Note: The job is a remote job and is open to candidates in USA. AgileEngine is an Inc. 5000 software company that creates award-winning software for Fortune 500 brands and startups across multiple industries, including application development and AI/ML. The Senior Data Engineer will architect, build, and scale a modern data platform by developing ETL/ELT pipelines, optimizing Snowflake data warehouse schemas, orchestrating workflows, integrating APIs and event-driven sources, and implementing data quality, observability, and DataOps practices.
Responsibilities
- Data Pipeline Development: Design, build, and maintain reliable, scalable ETL/ELT workflows that process batch and streaming data from diverse sources
- Data Warehousing & Modeling: Design efficient, production-ready schemas (normalized and denormalized) in Snowflake to optimize query performance and enable enterprise analytics
- API & Event Integration: Connect and ingest data from third-party REST APIs, event-driven streams, and batch sources into core data storage platforms
- Orchestration: Maintain and expand workflow orchestration pipelines using modern tools (Airflow, Prefect, or Dagster)
- Data Quality & Observability: Implement automated testing, validation, lineage tracking, and proactive alerting frameworks to guarantee data accuracy and system uptime
- DataOps & Engineering Standards: Drive CI/CD best practices, maintain code bases using Git and Docker, and adopt basic Infrastructure-as-Code (IaC) patterns
- Code Excellence: Apply modern software engineering standards—including design patterns, automated unit/integration testing, and clear documentation—to data repositories
Skills
- You must be authorized to work for ANY employer in the US (e.g., Green card holders, TN visa holders, GC EAD, H4 EAD, U4U with EAD), as we are unable to sponsor or take over employment visa sponsorship at this time
- 4+ years of experience as a Data Engineer
- Core Python Fundamentals: Demonstrable expertise writing modular, maintainable, and well-tested Python code (OOP/functional patterns, package management, standard testing frameworks)
- Advanced SQL & Modeling: Deep knowledge of complex SQL queries, query optimization, database design principles, and normalization/denormalization patterns
- Data Warehousing: Solid, hands-on experience building, managing, and optimizing data architectures within Snowflake
- Workflow Orchestration: Production experience using workflow orchestration engines like Apache Airflow, Prefect, or Dagster
- Integrations & Ingestion: Hands-on experience working with REST APIs, event-driven architectures, and both batch and streaming pipelines
- Data Quality & Lineage: Experience building automated data quality checks, data lineage, and alerting mechanisms (e.g., using tools like dbt test, Great Expectations, or similar)
- DevOps / DataOps Practices: Strong skills in version control (Git), containerization (Docker), CI/CD automation, and familiarity with Infrastructure-as-Code basics
- Upper-intermediate English level
- Experience with dbt (data build tool) for data transformations
- Familiarity with major cloud providers (AWS, GCP, or Azure)
- Exposure to message streaming tech like Apache Kafka or AWS Kinesis
Benefits
- Build your skills through mentorship, internal TechTalks, challenging projects, and a dedicated annual learning budget
- Regular performance and compensation reviews
- Work 100% remotely with flexible hours that support focus, autonomy, and a healthy work rhythm
- Build impactful products using modern technologies alongside global teams and leading brands
- A supportive environment with zero micromanagement where ideas are welcomed and contributions are recognized
- Access local well-being programs and people-focused support tailored to your location
Company Overview