Note: The job is a remote job and is open to candidates in USA. Innodata is a global data engineering company focused on enabling the responsible advancement of artificial intelligence through data, evaluation frameworks, platforms, services, and human expertise. The Data Platform Engineer will support, maintain, troubleshoot, and enhance large-scale production data platforms on Google Cloud, with an emphasis on platform reliability, performance optimization, production support, and continuous improvement.
Responsibilities
- Support and maintain production GCP data platforms, pipelines, and workflows across batch and streaming workloads
- Support production workflows orchestrated through Cloud Composer / Apache Airflow, including troubleshooting DAG failures, dependencies, scheduling issues, retries, and data-latency or SLA issues
- Optimize Airflow DAGs and orchestration workflows to improve reliability, execution time, recoverability, and operational efficiency
- Troubleshoot pipeline failures, data latency, performance degradation, configuration problems, and infrastructure-related issues
- Work extensively with Cloud Spanner, BigQuery, Dataflow, Pub/Sub, Cloud Composer (Apache Airflow), Cloud Storage, Cloud Monitoring and Logging, IAM, and related GCP services
- Monitor platform health and proactively improve reliability, scalability, performance, and operational efficiency
- Optimize existing pipelines and workloads to improve data refresh times, throughput, query performance, and platform stability
- Support platform, infrastructure, configuration, and dependency upgrades while maintaining production stability and compliance
- Perform root-cause analysis for recurring production issues and implement sustainable fixes
- Support IAM, access controls, monitoring, alerting, logging, and operational governance
- Collaborate with client and cross-functional Data Engineering, BI/Analytics, Application Engineering, Infrastructure, and Platform teams on production issues and enhancements
- Adapt to established client-specific engineering, security, compliance, review, and operational processes
- Review existing architectures and recommend incremental improvements without unnecessarily redesigning stable production systems
- Create and maintain technical documentation, operational runbooks, troubleshooting guides, and platform support procedures
Skills
- * Strong hands-on experience building, supporting, or maintaining production workloads on Google Cloud Platform (GCP)
- * Strong experience with BigQuery and Cloud Spanner
- * Hands-on experience with Dataflow, Pub/Sub, batch/streaming data pipelines, and Cloud Composer / Apache Airflow
- * Strong Python and SQL skills
- * Good understanding of GCP IAM, service accounts, permissions, monitoring, logging, alerting, and production operations
- * Experience troubleshooting complex production environments and performing root-cause analysis
- * Understanding of data ingestion, transformation, orchestration, data quality, performance optimization, and reliability
- * Ability to quickly understand existing systems, codebases, pipelines, configurations, and client-specific tools and workflows
- * Strong communication and collaboration skills across engineering and business teams
- * 3-7+ years of experience in Data Engineering, Cloud Engineering, Platform Engineering, or a related role
- * Experience supporting large-scale, business-critical data platforms with demanding availability, reliability, and data-freshness requirements
- * App Engine and other GCP-hosted applications
- * Cloud Run, GKE, or other GCP application services
- * CI/CD and DevOps practices on GCP
- * Infrastructure as Code, particularly Terraform
- * Data observability and automated data-quality monitoring
- * Supporting AI/ML, GenAI, or LLM-based applications running on GCP, including Vertex AI
- * Ability to quickly learn and operate within enterprise-specific analytics, engineering, and operational tooling
Company Overview
Company H1B Sponsorship