Experience: 10+ Years
Mandatory Requirement: EX Capital One Only
Job Description
We are seeking a highly experienced Senior Data Engineer with 10+ years of overall experience and prior Capital One experience. The ideal candidate will have strong expertise in designing, developing, and maintaining scalable data platforms, ETL/ELT pipelines, and cloud-based data solutions.
Responsibilities
Design, develop, and maintain scalable and reliable data pipelines and data processing solutions.
Build and optimize ETL/ELT workflows for large-scale structured and unstructured datasets.
Develop cloud-native data solutions, preferably using AWS services.
Work extensively with Python, SQL, Spark/PySpark, and distributed data-processing technologies.
Design and maintain data lakes, data warehouses, and enterprise data platforms.
Implement data quality, validation, monitoring, governance, and security best practices.
Optimize data pipelines and queries for performance, scalability, and cost efficiency.
Collaborate with data architects, analysts, data scientists, product teams, and business stakeholders.
Participate in architecture/design discussions and provide technical leadership and mentoring.
Apply experience and engineering practices gained from working within Capital One’s technology/data environment.
Required Skills
10+ years of Data Engineering experience
Previous Capital One experience – Mandatory
Strong Python and SQL
Strong Apache Spark / PySpark
Hands-on experience with AWS and cloud-based data engineering
Strong experience developing ETL/ELT and batch/streaming data pipelines
Experience with large-scale data processing and distributed systems
Experience with data lakes, data warehouses, and modern data architectures
Strong understanding of data modeling, data quality, governance, and security
Experience with CI/CD, Git, automated testing, and production deployment
Excellent communication, troubleshooting, and stakeholder-management skills
Flexible work from home options available.