Note: The job is a remote job and is open to candidates in USA. Vanna Health is hiring a Senior Data Engineer, Platform to own and evolve the infrastructure behind its healthcare data pipelines. The role leads platform architecture, ingestion, compute, orchestration, reliability, security, and making governed data usable by analytics teams and AI systems.
Responsibilities
- Shape the data platform roadmap with data leadership, own delivery of the work you lead, and keep stakeholders informed of progress, risks and trade-offs
- Set technical direction for data engineering: design and code-review standards, mentoring, and unblocking the engineers and contractors who work on the platform
- Own the platform architecture on Databricks and AWS, including ingestion, storage, compute, access control and environments. Write the design docs and make the build-versus-buy calls
- Own connectivity to partners and systems, built as tested, versioned software: payer claims, eligibility and attribution files over SFTP and S3, private networking, clinical and event feeds (FHIR, HL7), and data exchanged with our care platform
- Decide where and how jobs run, and make them fast, memory-predictable and economical: compute choices, serverless versus clusters, right-sizing, and moving single-machine Python onto distributed compute
- Own orchestration and deploys, and treat infrastructure and pipelines as code: Terraform, GitLab CI/CD, automated tests and release practices
- Make the platform reliable and observable, and turn failures into fixes that hold. Know that a job failed, quietly skipped, or ran twice as long as usual before anyone downstream notices, then ask what would have caught it earlier and what could have recovered it without you
- Make our data usable by AI: governed metric definitions, Genie spaces and MCP servers, so colleagues and AI agents work from trusted data rather than guesses
- Validate what arrived against what was promised: row counts, date spans, expected fields, and whether history reaches as far back as it should
- Replace recurring manual work with automation and AI agents, so routine data operations run themselves and people are pulled in only when something needs judgment
- Take part in a shared data operations rotation, which we keep light through automation
- Protect PHI by design: access controls, auditing, de-identification and secure data exchange, in partnership with our security leadership
- Give analytics engineering a platform to build on: they build the gold tier the analytics team uses, and you make sure it builds reliably, on time and at a sensible cost
- Represent data engineering with partners, vendors and colleagues, and share what you learn
Skills
- Typically seven or more years building production software or data systems, including several years owning data infrastructure or pipelines end to end, from design through operation
- Hands-on production experience with at least one distributed data processing engine at scale, such as Spark or Databricks, Snowflake, BigQuery, Flink, Dask or Ray, rather than single-machine pandas: you know what belongs on one machine and what does not, have made large jobs fast and memory-predictable, and can describe one with before and after numbers
- Strong Python, and SQL you can defend: not just that the query runs, but why you believe the answer it gives you, and what you checked before you believed it
- Experience designing systems on a major cloud, AWS preferred (storage, compute, networking, IAM and cost), with production engineering discipline: automated testing, CI/CD, infrastructure as code, code review, observability and incident response
- You have led technical work: set the direction for a project or a team, and mentored other engineers
- You have chased a data problem that turned out to be upstream of you, and you kept going until you found where it actually started
- You use AI coding agents in your daily engineering work, and can tell us about an AI result you caught being wrong on real data and how you caught it. Knowing when an answer is wrong, and checking before you act on it, is the harder half of that skill
- AWS preferred
- Databricks, Delta Lake, Unity Catalog and dbt
- Databricks Genie, MCP servers, or other ways of putting governed data in front of AI agents
- Healthcare data: claims (837/835), eligibility (834), FHIR, HL7 v2 and hospital event feeds, or another regulated setting such as HIPAA or SOC 2
- Workflow orchestration (Databricks Workflows, Temporal), event-driven ingestion and serverless compute
- Experience leading or managing engineers, or a clear interest in growing into a data engineering lead
Benefits
- Remote work
Company Overview