Note: The job is a remote job and is open to candidates in USA. HCLTech is seeking an AWS Data Quality Engineer to join its Data Engineering and Analytics team. The role focuses on building automated data validation frameworks, monitoring data quality across AWS-based platforms, testing ETL/ELT workflows, and supporting DataOps, observability, and data governance initiatives.
Responsibilities
- Develop automated validation frameworks to verify source-to-target data movement across enterprise data pipelines
- Create reusable Python-based testing utilities and automation scripts for validating schema consistency, completeness, accuracy, and timeliness of datasets
- Design and maintain data reconciliation processes for large-scale structured and semi-structured datasets
- Validate data pipelines built on AWS services including Amazon S3, AWS Glue, Amazon Redshift, and AWS Lambda
- Perform end-to-end testing of ETL/ELT workflows and cloud-native data architectures
- Ensure data transformation logic and business rules are correctly implemented throughout the data lifecycle
- Implement continuous monitoring solutions for:
- Data drift detection
- Schema validation
- Data anomaly identification
- Pipeline health monitoring
- Build and maintain data quality dashboards, alerts, and operational reports
- Establish proactive controls to detect and prevent data quality issues before they impact business stakeholders
- Partner with Data Engineers, BI Developers, Data Analysts, and Application Teams to translate business requirements into measurable data quality KPIs and SLAs
- Participate in DataOps processes and CI/CD deployments to ensure quality checks are integrated into delivery pipelines
- Support root-cause analysis and remediation of data quality issues
- Analyze large-scale datasets using SQL and Python to identify inconsistencies and trends
- Optimize validation frameworks for scalability and performance
- Contribute to data governance and quality best practices across the organization
Skills
- Candidates must be U.S. Citizens, U.S. Permanent Residents (Green Card Holders), or otherwise authorized to access export-controlled information under applicable U.S. regulations
- 2–3 years of experience in Data Engineering, Data Quality Engineering, ETL Testing, or related roles
- Experience working with enterprise-scale cloud data platforms
- Strong hands-on experience with:
- Amazon S3
- AWS Glue
- Amazon Redshift
- AWS Lambda
- Advanced proficiency in Python
- Experience building automation frameworks and validation utilities
- Experience with Pandas, PySpark, or equivalent data processing libraries
- Experience implementing data quality monitoring frameworks such as:
- Amazon Deequ
- Great Expectations
- Custom Python-based validation frameworks
- Knowledge of data profiling, reconciliation, and quality assurance methodologies
- Experience with Apache Airflow or similar orchestration tools
- Understanding of ETL/ELT architectures and data transformation processes
- Experience working with batch and distributed data processing environments
- Strong SQL skills for data validation and analysis
- Experience with UNIX/Linux shell scripting
- Understanding of data warehousing concepts and dimensional modeling
- Excellent analytical and problem-solving capabilities
- Strong communication and stakeholder management skills
- Ability to work independently in a fast-paced environment
- Strong attention to detail and commitment to data accuracy
- AWS Certified Data Engineer – Associate (Highly Preferred)
- CloudWatch (preferred)
- Experience with DataOps/DevOps practices
- Knowledge of CI/CD pipelines and automated testing integration
- Experience with data observability platforms and monitoring solutions
- Exposure to Agile/Scrum delivery environments
- EventBridge / Step Functions (good to have)
Benefits
- This role may also be eligible for performance-based bonuses subject to company policies.
- Medical insurance
- Dental insurance
- Vision insurance
- Pharmacy insurance
- Life insurance
- Accidental death & dismemberment insurance
- Disability insurance
- Employee assistance program
- 401(k) retirement plan
- 10 days of paid time off per year (some positions are eligible for need-based leave with no designated number of leave days per year)
- 10 paid holidays per year
- Continuous opportunities to find your spark and grow with us
- Transparent communication with senior-level employees
- Learning and career development programs at every level
- Opportunities to experiment in different roles or even pivot industries
- Unlimited opportunities to find the role that fits you best
Company Overview