Note: The job is a remote job and is open to candidates in USA. Deloitte is a professional services organization with an inclusive culture that empowers employees to contribute their perspectives and make a difference. The Databricks Engineer Contractor will support healthcare data engineering initiatives by building, optimizing, and maintaining scalable data pipelines and products using Databricks and Azure-based tools. The role also supports platform modernization, database solutions, data modeling, security controls, and production data quality.
Responsibilities
- Build relational database solutions based on project requirements
- Develop and maintain extract, transform, load (ETL) pipelines for warehouse and lakehouse environments
- Improve database and pipeline performance for efficient processing and retrieval
- Apply security controls and validation checks to protect data integrity and confidentiality
- Support data modeling, integration, and production data quality activities
Skills
- 3+ years using Azure Data Factory to build and schedule extract, transform, load (ETL) or extract, load, transform (ELT) pipelines
- 6+ years using PySpark for distributed data processing and transformation
- 3+ years using Python for scripting, pipeline development, and data validation
- 6+ years writing American National Standards Institute Structured Query Language (ANSI SQL) for joins, transformations, aggregations, and query tuning
- 6+ years using Git and GitHub for version control and pull request workflows
- 3+ years using Databricks and Delta Lake to build and maintain lakehouse pipelines
- 3+ years building data models for warehouse or lakehouse environments
- 3+ years using Apache Kafka for streaming or event-driven ingestion
- 1+ years using Prefect for workflow orchestration
- 1+ years using GitHub Actions or Azure Pipelines for continuous integration and continuous deployment (CI/CD)
- 1+ years working with Medallion Architecture
- 1+ years applying data quality, schema validation, and data governance controls in production pipelines
Benefits
- Remote work
Company Overview