Note: The job is a remote job and is open to candidates in USA. Xinova Group is presenting a contract opportunity with a leading healthcare organization focused on modernizing its enterprise data platform. The Databricks Data Engineer will design, develop, and optimize scalable Lakehouse architectures, pipelines, and governance frameworks for healthcare analytics, interoperability, AI, and machine learning initiatives while ensuring security and regulatory compliance.
Responsibilities
- Design, develop, and maintain scalable data pipelines using Databricks, PySpark, Spark SQL, and cloud-native services
- Build and optimize enterprise Lakehouse architectures leveraging Databricks, Delta Lake, Unity Catalog, and Apache Spark
- Develop robust ETL/ELT frameworks supporting healthcare analytics, clinical reporting, operational intelligence, and AI initiatives
- Design and implement ingestion pipelines for healthcare data sources including EHR, EMR, claims, pharmacy, laboratory, provider, and patient data
- Support integration and processing of healthcare interoperability standards such as HL7, FHIR, X12, and related healthcare data formats
- Develop and maintain Delta Live Tables, Databricks Workflows, and streaming data pipelines where applicable
- Implement data quality, lineage, observability, and monitoring frameworks across the Databricks platform
- Optimize Databricks workloads for performance, reliability, scalability, and cost efficiency
- Partner with business and clinical stakeholders to translate healthcare data requirements into scalable technical solutions
- Implement governance and security controls utilizing Unity Catalog and cloud-native security capabilities
- Ensure compliance with HIPAA, PHI, and healthcare data privacy requirements throughout the data lifecycle
- Support advanced analytics, machine learning, predictive modeling, and AI use cases utilizing Databricks capabilities
- Collaborate with DevOps and platform teams to establish CI/CD processes and Infrastructure-as-Code standards
- Troubleshoot complex data integration, performance, and platform issues across the healthcare data ecosystem
- Contribute to the organization's cloud modernization and healthcare data strategy initiatives
- Promote engineering standards, documentation, and continuous improvement across the data platform
Skills
- 5+ years of experience in Data Engineering, Big Data Engineering, or Cloud Data Platform development
- 3+ years of hands-on Databricks development experience in enterprise environments
- Strong expertise with Databricks Lakehouse Platform, Delta Lake, Apache Spark, PySpark, and Spark SQL
- Demonstrated experience building enterprise-scale data pipelines and cloud-native data platforms
- Advanced proficiency in Python and SQL
- Strong understanding of modern Lakehouse architecture principles and implementation best practices
- Experience with Databricks Unity Catalog, Delta Live Tables, and Databricks Workflows
- Experience utilizing Databricks for large-scale batch and streaming data processing workloads
- Hands-on experience with Azure, AWS, or Google Cloud Platform
- Strong knowledge of data modeling, data warehousing, and enterprise data architecture concepts
- Experience implementing data governance, security, and data quality frameworks
- Knowledge of healthcare compliance requirements including HIPAA and protected health information (PHI)
- Experience with CI/CD practices, DevOps methodologies, and Infrastructure-as-Code tools
- Excellent troubleshooting, analytical, and problem-solving skills
- Strong communication skills with the ability to collaborate across technical and business teams
- Active Databricks certifications (Databricks Data Engineer Associate or Professional preferred)
- Experience working within healthcare providers, payers, health systems, life sciences, or healthcare technology organizations
- Experience integrating healthcare data sources including Epic, Cerner, Athena, Meditech, or other EHR/EMR platforms
- Knowledge of healthcare interoperability standards including HL7, FHIR, CDA, and X12
- Experience supporting population health, clinical analytics, quality reporting, or value-based care initiatives
- Experience with MLflow, Databricks Asset Bundles, Unity Catalog, and Databricks Data Intelligence Platform capabilities
- Experience working with healthcare claims, patient, provider, clinical, laboratory, or operational datasets
- Experience with real-time and streaming data architectures
- Cloud certifications in Azure, AWS, or Google Cloud
- Experience supporting AI, machine learning, and generative AI initiatives within regulated environments
Benefits
- Remote work arrangement in Dallas, TX
Company Overview