Note: The job is a remote job and is open to candidates in USA. Fractal is a strategic AI partner to Fortune 500 companies, dedicated to enhancing human decision-making in enterprises. The Databricks Data Engineer will design, build, and optimize scalable data pipelines for Claims Payment Integrity analytics, ensuring high-quality data availability for various analytics teams.
Responsibilities
- Build scalable ETL/ELT pipelines in Databricks using PySpark, Spark SQL, Delta Live Tables, and workflows
- Engineer curated datasets across bronze/silver/gold layers for claims, pricing, provider, RCM, and member data
- Implement Delta Lake best practices including ACID transactions, schema evolution, CDC, and optimized storage formats
- Automate ingestion/transformation of large datasets from claims systems, provider files, call center platforms, and EHR feeds
- Work with Data Analysts, Data Scientists, and PI SMEs to translate analytic requirements into production data assets
- Support cluster optimization, table indexing (Z‑ORDER), and cost‑efficient lakehouse operations
- Participate in Agile ceremonies and ensure timely delivery of engineering tasks
Skills
- Hands-on experience with Databricks (PySpark, SQL, Delta Lake, Jobs/Workflows)
- Strong Spark performance tuning experience
- Experience engineering data for claims, provider, and membership domains
- Strong understanding of healthcare data models and adjudication flows
- Typically 5–8 years of Data Engineering experience in healthcare
- Bachelor's degree (4‑year)
- Experience with Call center data (member & provider interactions), Provider RCM datasets, and EHR/clinical data
- Experience with DLT, CI/CD, and MLflow‑integrated pipelines
- Exposure to actuarial or PI forecasting workflows
Benefits
- As a full-time employee of the company or as an hourly employee working more than 30 hours per week, you will be eligible to participate in the health, dental, vision, life insurance, and disability plans in accordance with the plan documents, which may be amended from time to time.
- You will be eligible for benefits on the first day of employment with the Company.
- In addition, you are eligible to participate in the Company 401(k) Plan after 30 days of employment, in accordance with the applicable plan terms.
- The Company provides for 11 paid holidays and 12 weeks of Parental Leave.
- We also follow a “free time” PTO policy, allowing you the flexibility to take time needed for either sick time or vacation.
- You may be eligible for a discretionary bonus for the current performance period.
Company Overview