Note: The job is a remote job and is open to candidates in USA. INFT Solutions, Inc is seeking a highly skilled AWS Data Engineer with strong experience in building scalable cloud-native data platforms and ETL pipelines on AWS. The ideal candidate will design and maintain ETL pipelines, implement data lake architectures, and collaborate with various stakeholders to deliver robust cloud data solutions.
Responsibilities
- Design, develop, and maintain scalable ETL pipelines using AWS Glue and Apache Spark (PySpark)
- Build and orchestrate data workflows using AWS Step Functions
- Design and implement S3 Data Lake architectures following AWS best practices
- Develop and deploy containerized applications on Amazon EKS using Kubernetes
- Build event-driven data processing solutions using Amazon SQS, AWS Lambda, and KEDA for auto-scaling
- Manage metadata using AWS Glue Data Catalog
- Implement data validation and governance using AWS Glue Data Quality
- Monitor applications and data pipelines using Amazon CloudWatch
- Optimize ETL jobs, Spark workloads, and Kubernetes deployments for performance and scalability
- Collaborate with architects, developers, DevOps, and business stakeholders to deliver robust cloud data solutions
- Participate in Agile ceremonies including sprint planning, code reviews, and production deployments
Skills
- 5+ years of experience in Data Engineering or Cloud Engineering
- Strong hands-on experience with AWS Glue
- Experience building Spark ETL pipelines using PySpark
- Experience with AWS Step Functions
- Strong knowledge of Amazon S3 Data Lake architecture and storage patterns
- Hands-on experience with Amazon EKS and Kubernetes
- Experience implementing event-driven architectures using Amazon SQS, AWS Lambda, and KEDA
- Experience with AWS Glue Data Catalog and Glue Data Quality
- Strong experience monitoring AWS workloads using Amazon CloudWatch
- Good understanding of distributed data processing and cloud-native application design
- Experience working in Agile/Scrum environments
- Python / PySpark
- SQL
- Terraform or CloudFormation
- Docker
- Git
- CI/CD (Jenkins, GitHub Actions, CodePipeline)
- Kafka
- Apache Airflow
- Delta Lake / Iceberg / Lake Formation
- IAM, VPC, CloudTrail
Company Overview