Note: The job is a remote job and is open to candidates in USA. Capgemini is seeking a Senior Data Engineer to develop and manage cloud-based data infrastructure on Google Cloud Platform. The role focuses on building ETL/ELT pipelines, optimizing performance and costs, mentoring engineers, collaborating on analytics and machine learning use cases, and implementing data quality and governance practices.
Responsibilities
- Build and optimize ETL/ELT pipelines using Dataflow (Apache Beam), Dataproc (Spark), Pub/Sub, and Cloud Functions
- Architect and manage BigQuery, Cloud Storage, and Data Fusion for efficient data ingestion, transformation, and storage
- Mentor junior engineers, review code, and establish data engineering standards, CI/CD, and monitoring (e.g., Cloud Monitoring, Logging)
- Optimize pipeline performance, query efficiency, and GCP cost management (e.g., slot reservations, partitioning)
- Work with data architects, scientists, and analysts to deliver self-service data platforms and support advanced analytics/ML use cases
- Implement data validation, lineage, and metadata management (e.g., Data Catalog, dbt)
Skills
- Proficiency in SQL, Python, Java/Scala, and distributed systems (Spark, Beam)
- Hands-on with BigQuery, Dataflow, Dataproc, Pub/Sub, Composer (Airflow), and Terraform
- Familiarity with CI/CD, Git, and infrastructure-as-code
- GCP Certifications (e.g., Professional Data Engineer)
- Experience with real-time processing, Vertex AI, or multi-cloud environments
Benefits
- Medical, dental, vision and retirement benefits
- Remote work
Company Overview
Company H1B Sponsorship