Job Title: Data Engineer Databricks / Scala / Spark / Kafka
Work Location: Richardson, TX | Indianapolis, IN | Tampa, FL | Hartford, CT | Phoenix, AZ | St. Louis, MO (Hybrid)
Long Term Contract
If you find this opportunity suitable, kindly share your updated resume. Also, please let me know a good time to connect with you for a quick discussion.
Looking forward to hearing from you!
Job Description
We are seeking an experienced Data Engineer with strong expertise in Databricks, Scala, Apache Spark, and Kafka to design, develop, and maintain scalable data processing and streaming solutions.
Required Skills
Strong hands-on experience with Databricks
Excellent programming skills in Scala
Strong experience with Apache Spark
Hands-on experience with Kafka and real-time data streaming
Experience designing and developing scalable data pipelines
Strong understanding of data processing, transformation, and integration concepts
Ability to troubleshoot and optimize distributed data processing applications
Good-to-Have Skills
Experience with Change Data Capture (CDC)
Knowledge of Databricks Unity Catalog
Strong SQL skills
Experience with data governance, security, and access management
Experience working with cloud-based data platforms
Key Responsibilities
Develop and maintain scalable batch and real-time data pipelines using Databricks, Spark, Scala, and Kafka
Build data ingestion and transformation frameworks for large-scale datasets
Implement and support CDC-based data pipelines where required
Optimize Spark jobs and Databricks workloads for performance and scalability
Work with Unity Catalog for data governance and access management
Develop complex SQL queries for data transformation and validation
Collaborate with data architects, developers, and business teams to deliver reliable data solutions
Troubleshoot production issues and ensure data pipelines meet performance and reliability requirements
Required Technical Skills
Databricks | Scala | Apache Spark | Kafka
Good to Have: CDC | Unity Catalog | SQL