Note: The job is a remote job and is open to candidates in USA. Dynata delivers first-party data that helps businesses gain precise insights, activate audiences, and measure impact. The Senior Data Engineer will design, build, and maintain enterprise lakehouse data pipelines, transformation layers, and data models while delivering scalable ELT/ETL solutions, ensuring data quality, troubleshooting incidents, and mentoring other engineers.
Responsibilities
- Design and implement scalable batch and streaming data pipelines using Apache Spark, Kafka, and Flink
- Build and maintain the Bronze/Silver/Gold medallion architecture within the lakehouse (Delta Lake / Iceberg)
- Develop and optimize complex SQL and PySpark transformations for large-scale datasets
- Integrate structured, semi-structured, and unstructured data sources into the lakehouse
- Collaborate with data architects to evolve the physical and logical data models
- Implement data quality checks and monitoring using Great Expectations or dbt tests
- Write Infrastructure-as-Code for pipeline environments (Terraform, Helm)
- Participate in code reviews and enforce engineering standards and best practices
- Troubleshoot pipeline failures, performance bottlenecks, and data incidents
- Mentor junior and mid-level data engineers and contribute to internal knowledge sharing
Skills
- 6+ years of data engineering experience with a track record of enterprise-scale delivery
- Expert proficiency in Python and SQL; PySpark experience required
- Hands-on experience with Apache Spark, Delta Lake, or Apache Iceberg
- Experience with orchestration tools: Apache Airflow, Prefect, or Dagster
- Strong knowledge of cloud data services: AWS Glue, Azure Data Factory, GCP Dataflow
- Proficiency with version control (Git), CI/CD pipelines, and containerization (Docker/Kubernetes)
- Experience with dbt (data build tool) for transformation layer management
- Bachelor's degree in Computer Science, Engineering, or related technical field
- Experience with Databricks, Snowflake, or Apache Hudi
- Knowledge of streaming architectures and Apache Kafka
- Certifications: Databricks Certified Data Engineer, AWS Data Analytics Specialty
Benefits
- A discretionary incentive program may be provided as part of the compensation package
- A full range of medical and other benefits, dependent on full-time employment status
Company Overview
Company H1B Sponsorship