Note: The job is a remote job and is open to candidates in USA. Optum Insight, part of Optum, improves the flow of health data and information to create a more connected healthcare system. The Senior Cloud Data Engineer will lead cloud data modernization initiatives, migrating legacy platforms and ETL/ELT processes to scalable Azure, Snowflake, and Databricks solutions. The role combines hands-on data engineering, solution architecture, technical leadership, security and governance, DevOps, and collaboration across healthcare analytics programs.
Responsibilities
- Provide technical leadership in design and engineering for the modernization of legacy data platforms, ETL/ELT processes, and enterprise data warehouses to new technologies in the Public Cloud (AWS/Azure)
- Review and analyze existing SSIS, SQL Server, and legacy ETL implementations and develop modernization strategies
- Design, develop, and implement cloud-native data solutions utilizing Azure Data Factory (ADF), Azure Data Lake Storage Gen2 (ADLS Gen2), Snowflake, Databricks, Azure cloud services
- Design and develop scalable ETL/ELT pipelines supporting batch and streaming workloads
- Create source-to-target mappings and implement data ingestion frameworks supporting structured and unstructured data
- Build and optimize cloud data warehouses, data marts, and lakehouse architectures
- Implement full and incremental load patterns, change data capture (CDC), and automated data validation processes
- Perform data profiling, data quality assessment, and root cause analysis for data issues
- Create Process Flows using diagraming tools like Visio
- Automate pipeline processes using scheduling tools like AirFlow
- Conduct ETL/ELT unit testing; participate in system and integration testing; identify and remedy solution defects
- Design and implement enterprise data warehouse solutions utilizing dimensional modeling concepts including Star schemas, Snowflake schemas, Fact and dimension design, Data pipeline architectures
- Collaborate with business and technical stakeholders to understand requirements and translate them into scalable solutions
- Implement appropriate data security, privacy, governance, and compliance controls
- Ensure compliance with organizational standards for data protection, auditability, and data quality
- Proven expertise in managing highly sensitive PHI/PII, claims, provider, clinical, and member data while supporting Risk Adjustment (HCC/RAF), quality measurement programs, and ensuring HIPAA compliance, data governance, privacy, and security standards
- Establish data lineage, monitoring, and operational support processes
- Architected HIPAA-compliant Azure and Snowflake-based healthcare analytics platforms integrating Claims, Membership, Provider, Clinical, Laboratory, Pharmacy, and Care Management data
- Implement CI/CD pipelines and DevOps practices using GitHub Actions and other modern delivery tools
- Support Agile software delivery practices and contribute across the full development lifecycle, including: Requirements, Design, Development, Testing, Deployment, Production support
- Lead the assessment of current-state data platforms and develop future-state cloud data architecture roadmaps aligned with business objectives and enterprise technology strategy
- Define and document solution architecture patterns, integration approaches, data flows, and technical design specifications for enterprise-scale modernization initiatives
- Create implementation roadmaps, effort estimates, technical dependencies, and migration strategies for large-scale cloud transformation programs
- Provide technical leadership for cloud data engineering initiatives
- Mentor junior engineers and contribute to engineering best practices
- Collaborate effectively in matrixed environments with business stakeholders, architects, analysts, developers, and project teams
- Proactively identify implementation risks, constraints, and improvement opportunities
- Self-driven, ownership mindset to navigate ambiguity, identify options to resolve constraints, mitigate implementation risks and define solutions to implementation challenges with minimal supervision
- Build partnership and rapport with client technical resources by demonstrating strong written and verbal communication, presentation, analytical and interpersonal skills
- Ability to work closely with Business Analysts and SMEs to understand business requirements
- Ability to work in a matrixed team environment with distributed responsibility across different teams
- Proactive and diligent in identifying and communicating scope, design, development issues and recommending potential solutions
Skills
- 6+ years of experience in designing, developing, and implementing enterprise data warehouse and analytics solutions on prem and on cloud
- 3+ years of hands-on cloud data engineering experience utilizing Azure cloud technologies
- 3+ years of experience working as part of data engineering teams, including offshore teams, through a product life cycle - requirements, design, development, testing and deployment
- Experience with Azure Data Factory (ADF), Databricks, Snowflake, ADLS Gen2, SQL Server, Python, Spark/Pyspark, ETL/ELT development
- Understanding of data warehousing concepts, dimensional modeling, data quality management, data governance, data pipeline architecture
- Experience designing and implementing healthcare data and analytics solutions, including large-scale cloud data modernization, data warehouse migration, platform transformation initiatives, HL7 and laboratory data processing frameworks, API-based integrations, clinical data standardization, and cloud-native healthcare interoperability platforms
- Expertise in healthcare interoperability and industry standards including HL7, FHIR, X12 EDI (837/835/834), ICD-10, CPT, HCPCS, and LOINC
- Comprehensive understanding of healthcare payer operations, CMS and NCQA regulatory frameworks, Population Health Management, Healthcare Economics, and FACETS-supported business processes across Claims, Membership, Eligibility, Provider, and Care Management domains
- Experience implementing CI/CD and DevOps practices using GitHub and GitHub Actions
- Experience working within Agile delivery methodologies
- Advanced AI/GenAI expertise with tools such as Databricks Genie, GitHub Copilot, Snowflake Cortex, LLMs, prompt engineering, Azure OpenAI, Databricks Mosaic AI
- Partner with AI Engineering on RAG pipelines, embedding strategies, and vector stores that ground responses in EHR data, clinical documentation, and medical literature, including implementing data refresh patterns that sync with EHR updates and evidence-based guideline revisions
- Ability to provide architectural guidance and mentor engineers
- Consulting-oriented background with experience aligning technology strategy to business outcomes, facilitating stakeholder discussions, defining migration approaches, and driving enterprise modernization programs
- Ability to accommodate 10 - 20% travel when required
- Snowflake, Azure, Databricks, or Cloud Data Engineering certification
- Experience with cloud data platforms across Azure, AWS, and/or Google Cloud
- Experience working with healthcare claims, member, enrollment, or clinical data
- Experience with streaming and event-driven architecture utilizing Kafka and Spark Structure Streaming
- Experience with Infrastructure as Code (Terraform) and container technologies (Docker, Kubernetes)
- Experience leading architecture assessments, developing modernization roadmaps, producing future-state architecture designs, and supporting executive-level technology strategy discussions
- Demonstrated ability to meet tight deadlines, follow development standards and effectively raise critical issues with the client
- Excellent analytical, problem-solving, communication, and stakeholder management skills
- Demonstrated ability to work independently, manage ambiguity, and deliver high-quality solutions
Benefits
- Flexibility to work remotely from anywhere within the U.S.
- A comprehensive benefits package
- Incentive and recognition programs
- Equity stock purchase
- 401k contribution
- Development for other roles you may be interested in
Company Overview
Company H1B Sponsorship