Our client is seeking an experienced Big Data Engineer to design, develop, and support large-scale data pipelines and transformation processes. The ideal candidate will have strong SQL, Python, ETL/ELT, and data engineering experience, along with the ability to communicate technical concepts clearly to clients, stakeholders, and cross-functional teams.
Java experience is helpful for supporting existing systems and integrations but is not the primary focus of this position.
Responsibilities
- Design, build, maintain, and optimize ETL/ELT pipelines that move and transform large-scale data across source and target systems.
- Write and optimize complex SQL queries, stored procedures, data models, and validation processes.
- Develop data transformation, integration, and automation solutions primarily using Python.
- Use Java as needed to support existing applications, services, and system integrations.
- Parse, transform, and validate XML-based data formats.
- Work within Linux and shell environments to support scripting, scheduled jobs, cron processing, log reviews, and troubleshooting.
- Implement data validation, error handling, monitoring, and traceability controls.
- Troubleshoot pipeline failures, data discrepancies, performance issues, and integration defects.
- Collaborate with client, security, infrastructure, data engineering, and delivery teams.
- Communicate technical issues, risks, and recommendations clearly to technical and nontechnical stakeholders.
- Participate in Agile planning, status reporting, documentation, issue tracking, and cross-team coordination.
Required Qualifications
- Bachelor’s degree from an accredited college or university or equivalent professional experience.
- Five or more years of professional experience in data engineering, database development, ETL/ELT development, or a related data integration role.
- Strong hands-on SQL experience, including complex queries, performance tuning, stored procedures, and data modeling.
- Experience designing, building, optimizing, and supporting large-scale data pipelines.
- Strong Python programming experience supporting data transformation, integration, automation, or pipeline operations.
- Experience parsing, transforming, and validating XML-based data.
- Experience working in Linux and shell environments, including scripting, scheduled jobs, cron, and log troubleshooting.
- Experience implementing data validation, error handling, pipeline monitoring, and production troubleshooting.
- Strong written and verbal communication skills.
- Ability to explain technical concepts clearly to clients and stakeholders.
- Ability to collaborate across security, infrastructure, engineering, and delivery teams.
- Self-motivated and proactive, with a positive and collaborative approach.
- U.S. citizenship is required.
- Must be willing and able to undergo a background investigation for a Public Trust suitability determination.
Preferred Qualifications
- Experience using Python as the primary language for data pipeline automation, transformation, monitoring, or production troubleshooting.
- Experience supporting Java-based applications, services, or integrations.
- Working knowledge of AWS data services, including S3, Glue, Lambda, RDS, or Redshift.
- Exposure to XBRL or other structured financial, regulatory, or compliance-driven data formats.
- Experience supporting federal, government, or compliance-driven data environments.
- Experience with Oracle, PostgreSQL, Redshift, or other relational database platforms.
- Familiarity with reporting, analytics, or dashboarding tools such as Power BI or MicroStrategy
Originally posted on