Note: The job is a remote job and is open to candidates in USA. RSNGINFO SOLUTIONS is seeking a Lead Databricks Developer to support healthcare data engineering initiatives. The role focuses on designing scalable ETL/ELT pipelines, integrating healthcare data sources, optimizing Databricks workflows, and ensuring data quality, security, and regulatory compliance.
Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks, PySpark, and Spark SQL
- Build high-performance data solutions using Delta Lake and Medallion Architecture (Bronze, Silver, Gold)
- Develop batch and streaming data pipelines from multiple healthcare data sources
- Process and transform healthcare datasets, including Claims, Member, Provider, Enrollment, Pharmacy, and Clinical data
- Integrate data from EHR/EMR systems, HL7/FHIR APIs, and healthcare data warehouses
- Optimize Spark jobs for performance, scalability, and cost efficiency
- Develop reusable notebooks, workflows, and job schedules within Databricks
- Collaborate with Data Architects, Data Analysts, Business Analysts, and healthcare stakeholders to deliver high-quality solutions
- Implement data quality checks, validation rules, and monitoring for production pipelines
- Ensure compliance with healthcare regulations, including HIPAA, PHI, and data governance standards
- Support production deployments, troubleshoot pipeline failures, and optimize existing data workflows
Benefits
- Remote work
Company Overview