Note: The job is a remote job and is open to candidates in USA. iFlow Inc. is hiring a Senior Data Engineer to support the migration and modernization of its Vehicle Catalog data platform on Databricks. The role focuses on refactoring ETL/ELT pipelines, migrating datasets and reporting assets, validating data parity, and updating operational support processes. The engineer will collaborate with data engineers, analysts, and other stakeholders throughout the migration program.
Responsibilities
- Work with one of our Staff Data Engineers to understand the migration plan, sequence your work against it, and provide regular progress updates
- Refactor existing ETL/ELT code written in PySpark, DABs and dbt from the current Databricks environment to meet the patterns and standards of the new account
- Migrate data pipelines and their underlying datasets, including backfills, and validate parity between source and target
- Build and run reconciliation checks row counts, schema conformance, freshness, completeness to confirm migrated data matches the legacy environment
- Migrate dashboards and downstream reporting assets to point at the new datasets, coordinating with the analysts who own them
- Update orchestration, scheduling, alerting, and runbooks so migrated pipelines are supportable after cutover
- Troubleshoot pipeline failures and data discrepancies during and after cutover
- Document what was migrated, what changed, and anything the team needs to know to own it going forward
- Take on adjacent migration and data engineering requests as the program evolves
Skills
- 6+ years of experience in Data Engineering, Analytics Engineering, or Software Engineering working with production data systems
- Strong expertise in SQL and Python for building scalable data pipelines and transformations
- Strong familiarity with AI tools like Claude Code / Devin for SW development process
- Hands-on experience building ELT pipelines using modern cloud data platforms (Databricks experience required)
- Building robust testing frameworks for Data Migration ( Completeness, Quality etc)
- Deep experience with dbt, including model development, testing, documentation, and CI/CD integration
- Experience designing and operating production-grade data pipelines with monitoring and observability
- Strong collaboration skills and ability to partner with engineering teams, analysts, and operational stakeholders
- Bachelor s or master s degree in computer science, Engineering, Mathematics, or related field, or equivalent practical experience
- Prior data or platform migration experience
- Familiarity with Fivetran connector management and ingestion architecture
- Experience with data contracts and schema governance
- Knowledge of streaming or near-real-time data pipelines
- Would prefer someone in the Pacific, Mountain, or Central time zone in that order, if possible
Company Overview
Company H1B Sponsorship