Note: The job is a remote job and is open to candidates in USA. CVS Health is seeking a Staff Data Engineer to lead the architecture, implementation, and operational excellence of its Azure Databricks platform. The role focuses on platform reliability, security, governance, performance, automation, and enabling data engineering teams, while directing two platform engineers.
Responsibilities
- Operational Excellence: Oversee the operational health of the platform. Maintain and enhance monitoring, logging, and alerting solutions to proactively identify and resolve issues. Lead incident response and perform root cause analysis for platform-wide problems
- Platform Architecture & Strategy: Maintain and enhance the core components of our Azure Databricks environment. Define the strategic roadmap for the platform, including the adoption of new features, services, and best practices (e.g., Delta Lake, Unity Catalog, Databricks Workflows. Github CI/CD)
- Security & Governance: Manage the enforcement and enhancement of data security and governance policies. Manage the Unity Catalog to ensure fine-grained access control, data lineage, and compliance with internal and external regulations
- Performance & Cost Optimization: Implement strategies for cost management and performance tuning at a platform level. Provide guidance and tooling to optimize Databricks clusters and workloads to maximize efficiency and minimize cloud spend
- Tooling & Automation: Maintain and enhance automated solutions for provisioning, scaling, and managing Databricks workspaces and resources using Infrastructure as Code (Terraform) and CI/CD pipelines (GitHub)
- Developer Enablement: Serve as a subject matter expert and mentor to other data engineers and Azure databricks platform engineers. Develop and advocate for best practices, create reusable patterns and libraries, and provide expert-level technical support and training to data teams
- Cross-Functional Collaboration: Partner with data engineering and business teams to understand their needs and maintain and enhance a platform that empowers them. Work with security, networking, and other infrastructure teams to ensure a cohesive and secure environment
- Leading the Platform Engineers: Overseeing and directing the efforts of 2 platform engineers
Skills
- 8+ years of experience in a hands-on engineering role with a focus on cloud data platforms, distributed systems, and data engineering
- Deep, hands-on experience with the Azure Databricks platform, including its administration, cluster management, job orchestration, and key features like Delta Lake and Unity Catalog
- Expert-level knowledge of the Azure ecosystem, including Azure Data Lake Storage (ADLS), Azure Active Directory (AAD), and networking components
- Strong proficiency in Terraform, Python, Apache Spark/PySpark, SQL, and Shell scripting
- Proven experience with Infrastructure as Code tools (e.g., Terraform) and building automated CI/CD pipelines for platform and data application deployments
- A strong understanding of distributed computing principles and the architecture of large-scale data systems
- Exceptional analytical and problem-solving skills with a track record of diagnosing and resolving complex, platform-level issues
- Excellent communication, documentation, and presentation skills, with the ability to influence technical and business stakeholders
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field
- Experience with other data technologies (e.g., Azure Synapse, Kafka, Snowflake)
- Familiarity with healthcare data and healthcare insurance feeds is a significant plus
- Relevant certifications such as Microsoft Certified: Azure Solutions Architect Expert, Databricks Certified Data Engineer Professional, and Databricks Certified Platform Administrator
Benefits
- Eligible for a CVS Health bonus, commission or short-term incentive program in addition to the base pay range listed above.
- An award target in the company’s equity award program.
- Eligible for a comprehensive benefits package designed to support the physical, emotional, and financial well-being of colleagues and their families, including medical, dental, and vision coverage, paid time off, retirement savings options, wellness programs, and other resources, based on eligibility.
Company Overview
Company H1B Sponsorship