Note: The job is a remote job and is open to candidates in USA. The Judge Group is recruiting for a federal government client supporting a human-capital data modernization program. The Databricks Data Engineer builds and tests data pipelines, transformations, migration workflows, quality controls, APIs, curated datasets, reconciliation routines, and reusable data products.
Responsibilities
- Build and maintain Databricks ingestion, transformation, orchestration, migration, and data-processing pipelines
- Develop scalable ETL/ELT solutions using Databricks, Apache Spark, PySpark, Python, SQL, and Delta Lake
- Implement source-to-target mappings, transformation rules, business logic, data contracts, metadata, lineage, and data-quality requirements
- Develop migrated data feeds, APIs, curated datasets, reporting layers, and reusable data products
- Create reconciliation routines and support source-to-target validation, exception analysis, and defect resolution
- Build unit, integration, regression, performance, and data-quality tests and document technical results
- Support Git-based development, CI/CD, release management, deployment, monitoring, troubleshooting, and stabilization
- Analyze and resolve job failures, cluster-performance issues, pipeline bottlenecks, schema changes, and data-quality problems
- Collaborate with data architects, engineering leads, integration engineers, profiling analysts, testers, security specialists, governance leads, and source-system teams
- Document reusable code, technical patterns, runbooks, troubleshooting procedures, and lessons learned for later modernization waves
- Participate in Agile ceremonies, backlog refinement, sprint planning, demonstrations, and technical reviews
Skills
- Bachelor's degree in computer science, data engineering, information systems, engineering, analytics, or a related field
- Minimum 4 years of experience in data engineering, software development, ETL/ELT, cloud analytics, or a related technical discipline
- Hands-on experience with Databricks, Apache Spark, PySpark, Python, SQL, Delta Lake, and workflow orchestration
- Experience building, testing, and maintaining batch data pipelines, transformations, and curated data layers
- Experience with enterprise data migration, source-to-target mapping, reconciliation, validation, and data-quality controls
- Familiarity with Git, CI/CD, automated testing, deployment, monitoring, logging, and troubleshooting
- Strong analytical, problem-solving, documentation, collaboration, and communication skills
- Ability to obtain and maintain a Federal Public Trust
Benefits
- Healthcare
- 401k
- Medical, dental, and vision insurance are available to qualified candidates who meet eligibility requirements.
Company Overview