Note: The job is a remote job and is open to candidates in USA. DetailPage helps Amazon brands grow organic traffic through content optimization and market insights. The Data Engineer will build and optimize scalable data pipelines, support ETL/ELT and API processes, and ensure data flows reliably across the organization and to end users.
Responsibilities
- Design, develop, and maintain scalable data pipelines using a variety of AWS services
- Work with relational databases such as Aurora (PostgreSQL), Redshift, and AWS Glue for large-scale data management
- Develop and optimize ETL/ELT processes for smooth data integration from multiple sources
- Collaborate with cross-functional teams to ensure data accessibility and integrity across platforms
- Use Python, SQL, and Pandas for data manipulation, analysis, and workflow automation
- Implement unit testing (PyTest) to ensure code reliability
- Manage API integrations and development using FastAPI
- Assist with infrastructure as code (IaC) using CDK and Terraform
- Build CI/CD pipelines in Jenkins
Skills
- SQL mastery – expertise in complex SQL queries and database optimization
- 2-3 years of experience in Data Engineering with AWS and Python
- Linux proficiency – comfortable with shell scripting, common CLI tools, and building/testing Linux-based Docker images
- Advanced Python – strong hands-on experience, ideally with Data Engineering tools like PySpark, Pandas, and SQLAlchemy
- Git proficiency – comfortable with version control, branching, and collaboration
- ETL/ELT experience – proven ability to build and optimize extract, transform, and load processes
- Unit testing – familiarity with PyTest or similar tools
- AWS experience with Aurora (PostgreSQL or MySQL), Redshift, Fargate, Lambda, SQS, and IAM, plus tools like Boto3 and the AWS CLI
- Airflow expertise, particularly AWS Managed Airflow, for scheduling and orchestrating data workflows
- API development experience with API Gateway and FastAPI
- Infrastructure as Code experience with CDK or Terraform
- Familiarity with caching tools like Redis
- Basic DevOps knowledge for CI/CD pipelines and dev workflow optimization
- Familiarity with DBT
Company Overview