Note: The job is a remote job and is open to candidates in USA. Dahl Consulting is a staffing firm partnering with a leading enterprise retail and consumer technology organization. The Hadoop Platform Engineer will build, maintain, optimize, and support a large-scale Hadoop platform for enterprise data processing and analytics, while collaborating with data engineering, infrastructure, and platform teams.
Responsibilities
- Design, build, and maintain a reliable, scalable, and high-performing Hadoop platform
- Administer and optimize core Hadoop ecosystem components, including HDFS, Hive, Spark, YARN, and MapReduce
- Troubleshoot and resolve cluster-level performance issues through deep analysis of Hadoop and Java-based systems
- Develop and standardize monitoring, observability, and platform health frameworks
- Implement proactive monitoring, automated health checks, and alerting capabilities to identify and resolve issues before they affect end users
- Analyze and optimize Linux-based infrastructure to improve performance, resource utilization, and platform stability
- Collaborate with data engineering, platform, and infrastructure teams to improve workload efficiency and cluster performance
- Establish best practices for platform management, capacity planning, performance tuning, and upgrade strategies
- Automate operational processes and reduce manual effort through scripting and infrastructure automation
- Support data security, governance, and compliance requirements across the Hadoop ecosystem
- Drive root cause analysis efforts for production incidents and implement preventative solutions
- Create and maintain operational documentation, configuration standards, and support runbooks
- Manage large-scale environments using configuration management and automation tools
Skills
- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field, or equivalent practical experience
- Professional experience supporting and administering enterprise Hadoop platforms
- Strong hands-on experience with: + Apache Hadoop + Hadoop Distributed File System (HDFS) + Apache Hive + Apache Spark + Java + Linux administration + Ansible + Chef
- Experience diagnosing and optimizing Hadoop cluster performance
- Knowledge of monitoring, observability, and alerting frameworks
- Experience managing large-scale distributed systems and data platforms
- Strong troubleshooting and root cause analysis skills
- Experience with infrastructure automation and operational process improvement
- Ability to collaborate effectively across engineering and infrastructure teams
- Strong written and verbal communication skills
- Experience with Grafana for monitoring and visualization
- Experience developing automation or tooling using Python
- Experience with Trino and distributed query engines
- Experience with Bash or Scala scripting
- Experience supporting enterprise-scale analytics and data engineering environments
- Knowledge of data governance, security, and compliance practices within big data ecosystems
- Experience with capacity planning and platform modernization initiatives
Benefits
- A comprehensive benefits package to eligible employees that will allow you to choose the best coverage to meet your family’s needs.
Company Overview
Company H1B Sponsorship