Note: The job is a remote job and is open to candidates in USA. CyberCoders is seeking a Senior Data Engineer to support enterprise data initiatives in a healthcare and regulated data environment. The role leads the design, development, and optimization of secure cloud-based data platforms, pipelines, lakehouse architectures, analytics infrastructure, and AI-enabled solutions.
Responsibilities
- Design and develop scalable batch and real-time data pipelines using Databricks, Spark, Delta Lake, and AWS services
- Lead the evolution of a centralized lakehouse platform, including data modeling, governance, optimization, metadata management, and metrics layers
- Build and support ETL/ELT workflows utilizing Airflow, dbt, AWS Step Functions, and modern orchestration frameworks
- Develop streaming data solutions using Kafka and Kinesis for high-volume event processing
- Partner with Data Science and AI teams to support feature stores, model scoring pipelines, and machine learning enablement
- Build infrastructure supporting NLP and generative AI applications, including vector databases, document ingestion pipelines, knowledge repositories, and RAG-based architectures
- Develop APIs and data integrations that support business applications, analytics platforms, CRM systems, and external data exchange
- Implement data security, governance, privacy controls, audit logging, and regulatory compliance standards
- Design and maintain enterprise reporting infrastructure supporting automated report generation and multi-format delivery
- Conduct code reviews, mentor junior engineers, promote engineering best practices, and contribute to technical roadmap planning
Skills
- - Bachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, or a related technical discipline
- - 7+ years of professional data engineering experience
- - 3+ years working with cloud-native data lake or lakehouse architectures
- - Experience building and supporting large-scale enterprise data platforms
- - Strong expertise with Databricks , including Delta Lake, Unity Catalog, Spark, and Workflows
- - Advanced experience with Apache Spark using PySpark, Scala, or Java
- - Strong AWS experience including S3, Redshift, Athena, Glue, Lambda, Step Functions, Kinesis, DynamoDB, EMR, Bedrock, Transcribe, and Comprehend Medical
- - Proficiency in Python and SQL; Java or Scala experience is a plus
- - Experience with Airflow, dbt, and cloud-based orchestration frameworks
- - Knowledge of Kafka, Kinesis, and event-driven architectures
- - Experience building feature stores, ML data pipelines, and model-serving infrastructure
- - Familiarity with vector databases, embedding stores, document processing pipelines, and RAG architectures
- - Experience with Docker, Kubernetes/EKS, CI/CD pipelines, and infrastructure automation
- - Knowledge of data quality frameworks, governance standards, and data contract methodologies
- - Experience integrating BI platforms and developing reporting and analytics delivery solutions
- - Strong technical leadership and mentoring capabilities
- - Ability to architect scalable and reliable data solutions that support analytics, machine learning, and AI initiatives
- - Excellent collaboration skills with Data Science, Product, Engineering, and Analytics teams
- - Strong problem-solving mindset and passion for emerging data and AI technologies
- - Ability to balance innovation with reliability, security, compliance, and operational excellence
- Must have healthcare/HIPAA experience and must have Databricks experience
- For this position, you must be currently authorized to work in the United States without the need for sponsorship for a non-immigrant visa
- - Prior experience in healthcare, health technology, insurance, financial services, or other regulated industries preferred
Benefits
- Medical/Dental/Vision
- 401K with 40% match up to first 6% of associated contributions
- PTO, Holidays, Personal Days, Community Service Day
Company Overview