Note: The job is a remote job and is open to candidates in USA. Maganti IT Resources, LLC is seeking a Lead Cloudera/Hadoop Migration Engineer. The ideal candidate will possess deep expertise in both on-premises Hadoop ecosystems and modern cloud-native big data solutions on AWS, focusing on migrating data and applications to AWS services.
Responsibilities
- Analyze existing Cloudera Hadoop architecture, workflows, and data pipelines
- Develop a comprehensive migration strategy to transfer data, processing logic, and applications to AWS services
- Design target architecture using AWS big data services such as Amazon EMR, Airflow, Glue, Lambda
- Evaluate and select appropriate AWS tools for data ingestion, storage, processing, and analytics
- Oversee the migration process, including data transfer, application re-platforming, and validation
- Ensure the new architecture meets performance, scalability, reliability, and security requirements
- Collaborate with developers, data engineers, DevOps, and stakeholders to ensure a seamless migration
- Create detailed technical documentation and migration runbooks
- Mentor team members on AWS big data best practices
- Strong understanding of data security, compliance, and governance in the cloud
- Proven track record of migrating or modernizing big data workloads to AWS
- Experience with cloud automation and Infrastructure as Code (e.g. Terraform)
Skills
- Bachelor's Degree in Computer Science, Information Technology, Information Systems, or related field
- 10+ years of experience with enterprise and application design patterns that are applied on Hadoop and Big data platforms in aws
- Deep expertise in both on-premises Hadoop ecosystems and modern cloud-native big data solutions on AWS
- Hands-on experience with Cloudera Hadoop stack (HDFS, Hive, Spark, Oozie, Python)
- Proficiency in writing, maintaining, and troubleshooting shell scripts (Bash or similar) for automation and operational support
- Analyze existing Cloudera Hadoop architecture, workflows, and data pipelines
- Develop a comprehensive migration strategy to transfer data, processing logic, and applications to AWS services
- Design target architecture using AWS big data services such as Amazon EMR, Airflow, Glue, Lambda
- Evaluate and select appropriate AWS tools for data ingestion, storage, processing, and analytics
- Oversee the migration process, including data transfer, application re-platforming, and validation
- Ensure the new architecture meets performance, scalability, reliability, and security requirements
- Collaborate with developers, data engineers, DevOps, and stakeholders to ensure a seamless migration
- Create detailed technical documentation and migration runbooks
- Mentor team members on AWS big data best practices
- Strong understanding of data security, compliance, and governance in the cloud
- Proven track record of migrating or modernizing big data workloads to AWS
- Experience with cloud automation and Infrastructure as Code (e.g. Terraform)
- AWS certifications (e.g., AWS Certified Solutions Architect, AWS Certified Big Data – Specialty)
Benefits
- Fully Remote
- Need Only Visa independent Candidates who can work on our w2 or on 1099
Company Overview