Note: The job is a remote job and is open to candidates in USA. Optimal is seeking a Senior Data Engineer to own and evolve an organization-wide data platform for an established consumer marketplace. The role combines end-to-end architecture ownership with hands-on pipeline development, production debugging, platform reliability, cost efficiency, observability, and AI-enabled data engineering workflows.
Responsibilities
- Own and evolve data pipeline architecture across core domains - ingestion, transformation, modeling, serving
- Lead platform-level improvements: warehouse cost management, compute efficiency, access control
- Surface and lead technical initiatives (orchestration, CI/CD, developer experience) before they become blockers
- Drive large, complex projects spanning multiple teams, owning outcomes rather than just execution
- Own monitoring and testing strategy - close observability gaps and build alerting ahead of failures
- Mentor other data and analytics engineers, reviewing architectural and modeling decisions
- Integrate AI meaningfully into data engineering workflows, building leverage for the whole team
Skills
- Must hold existing US work authorisation - no sponsorship or transfer, now or in future including F1 Visas (US residents only)
- End-to-end data platform ownership, Python, strong SQL, pipeline orchestration, cloud data warehouse, containerised infra (Non-Negotiable)
- Python and strong hands-on SQL (deep enough to hold up in a live technical round)
- Pipeline orchestration in production (Airflow or equivalent - Prefect, Dagster)
- Cloud data warehouse in production (BigQuery or equivalent - Snowflake, Databricks)
- Containerised data infrastructure in production (Docker / Kubernetes)
- Built or improved CI/CD practices for data pipelines
- Must have startup experience - you've built in a fast-moving, high-ownership environment
- A blend of established mid-sized tech AND startup (leaning away from big-tech-only profiles)
- Owned data platform architecture end-to-end, not just implementing pipelines to spec
- Comfortable making architectural decisions independently and owning outcomes, not just execution
- Hands-on-keyboard - still shipping code and debugging production, not whiteboard-only
- 6-9 years as a data engineer or data platform engineer
- Owned a broad data platform end-to-end at whole-company scope (not a single pipeline or report)
- Treats reliability, observability, and cost efficiency as design constraints, not afterthoughts
- Able to explain complex technical concepts clearly to non-technical stakeholders
- Ability to work US hours
- Led a legacy ETL to modern orchestration migration end-to-end
- Hands-on Kubernetes in production
- Experience with AI-assisted tooling inside data workflows
Benefits
- Competitive equity
- Fully remote (US based)
- Strong work-life balance
- A real "learn it all" culture
Company Overview