Note: The job is a remote job and is open to candidates in USA. Huron is hiring a Data Engineer to operate and support clinical data pipelines and client-facing applications across multiple client environments. The role focuses on application deployments, data ingestion and validation, AWS infrastructure, CI/CD, monitoring, and incident response under the direction of the DataOps lead.
Responsibilities
- Support daily operation and monitoring of data pipelines across 10+ concurrent client environments, triaging failures (no-data, bad-data, and processing errors) and escalating complex issues to the DataOps lead before they affect client-facing reporting
- Set up, configure, and support deployment of client-facing applications — from environment setup through go-live stabilization and ongoing support
- Validate that daily data flows complete successfully across all client environments — at a console/log level day to day, diving deeper into pipeline and application logs when something doesn't look right
- Support provisioning and configuration of new client environments — SFTP ingestion setup, instance provisioning, access controls, and coordination with client IT teams on file formats, schedules, and encryption — partnering with the DataOps lead on more complex setups
- Contribute to the team's CI/CD pipelines (Jenkins), AWS infrastructure (EC2, S3, Lambda, EventBridge, IAM), and Posit Connect analytics hosting platform, under the DataOps lead's direction
- Build and maintain monitoring and alerting for pipeline health — distinguishing failure types, reducing false alarms, and keeping the team ahead of client-visible data issues
- Maintain runbooks and documentation so operational knowledge is transferable, not siloed in one person's head
- Help set up and configure client application environments ahead of deployment, including app and infrastructure configuration
- Run and support production deployments, including go-live stabilization and post-launch monitoring
- Validate that daily data loads and refreshes complete successfully for every client application — console-level checks as the first pass, with deeper investigation when volumes or completion look off
- Investigate application and data load issues when flagged, diagnosing root cause across the pipeline and application layers, with support from senior team members on complex cases
- Build and maintain client-specific data ingestion pipelines (SFTP, S3, event-driven triggers, file validation), with architectural guidance from the DataOps lead
- Build and maintain standardized data validation scripts — a reusable QA framework applied across client environments
- Ingest historical data for new client onboardings and resolve data quality incidents (missing files, null values, file extension issues, reprocessing)
- Support the team's upcoming migration of the data lake and core tables to **Snowflake**, the next major infrastructure initiative following this year's Athena consolidation
- Operate and monitor the daily processing pipeline across all client environments; distinguish lambda failures from regular pipeline failures to reduce false alarms
- Build data integrity alerts (e.g., null DRGs, late file delivery, failed runs) and route them to the team in real time
- Investigate and resolve daily processing failures — root cause analysis, temporary fixes, and permanent remediation
- Manage AWS Lambda functions — version control, deployment, and event-driven triggering (SNS/SQS)
- Build data volume QC dashboards to track pipeline health across clients
- Help manage AWS infrastructure (EC2, S3, RDS, Athena, IAM, Lambda, SNS/SQS) across all client deployments, under the direction of the DataOps lead
- Support Posit Connect administration — configuration, user access, and runtime management for the team's R-based reporting stack
- Provision new client instances — database setup, ingestion configuration, and access controls
- Handle SAML/SSO certificate management and PGP encryption setup for secure client file delivery
- Help enforce code repository governance — branch protection, PR approval requirements, and deployment controls
- Build and maintain Jenkins-based CI/CD pipelines (dev → staging → production)
- Set up and validate development and staging environments
- Maintain runbooks for all operational procedures
- Support the team's Docker-based mono repo deployment architecture across client environments
- Execute and document client offboarding processes
- Support Huron mandatory compliance activities, PHI data classification, and audit control documentation
- Manage certificate renewals, access restriction enforcement, and credential rotation (SMTP, IAM)
Skills
- U.S. work authorization is required
- Bachelor's degree in Computer Science, Information Systems, Engineering, or related field (or equivalent experience)
- 2-3+ years of experience in data engineering, DevOps, or DataOps roles, including hands-on experience supporting production data pipelines
- Strong hands-on experience with **AWS data pipeline infrastructure** — S3, Athena, Lambda, EC2, and event-driven architectures (SNS/SQS)
- Strong **Python** scripting ability for data ingestion, validation, and pipeline automation; comfort with **Bash**
- Experience with **CI/CD pipelines** (Jenkins or equivalent) and Git-based version control practices (branch protection, PR workflows)
- Experience setting up, configuring, and supporting application deployments — not just writing pipeline code
- Experience managing multi-client or multi-tenant data environments, including onboarding and provisioning new environments
- Experience participating in incident response and root cause analysis for data pipeline and application issues — this is an operations-heavy role, not a build-only role
- Comfortable working as a generalist across infrastructure, data engineering, and DevOps simultaneously, in a fast-moving, client-driven environment where priorities can shift mid-sprint
- Experience with **Snowflake** or other modern cloud data warehouse platforms — the team is migrating its data lake and core tables to Snowflake as its next major infrastructure initiative
- Posit Connect or R/Shiny environment familiarity
- Docker and containerized deployment experience
- SAML/SSO certificate management and cloud security practices (IAM key rotation, PGP encryption)
- Healthcare data environment experience — SFTP clinical file delivery, PHI handling, HL7/ADT familiarity
- Experience building monitoring and alerting systems for data pipelines
- Occasional direct client IT coordination experience (file delivery issues, certificate renewals, data quality questions)
Benefits
- Eligible to participate in Huron’s annual incentive compensation program.
- Medical, dental and vision coverage.
- Other wellness programs.
- Remote work.
Company Overview