Note: The job is a remote job and is open to candidates in USA. EPAM Systems is seeking a Lead Data Software Engineer to combine hands-on technical expertise with leadership responsibility in shaping scalable data architecture and delivering critical projects. The role focuses on building and maintaining ETL, batch, micro-batch, and streaming systems, while providing technical direction, coordinating stakeholders, managing delivery risks, and overseeing production support and deployments.
Responsibilities
- Develop and preserve the codebase supporting ETL and ELT pipelines, together with large batch, micro-batch, and streaming systems
- Set up the infrastructure required to pull, transform, and load data efficiently from a variety of sources using ADF, Spark, Kafka, or comparable technologies
- Identify, plan, and roll out internal process improvements, such as automating manual tasks, improving data delivery, and modifying infrastructure to increase scalability
- Architect and release data services solutions using technologies such as Spring Boot, ReactJS, NoSQL, and other related UI and API tools
- Maintain governance across delivery management and production processes, aligned with the chosen delivery model
- Act as the key contact for delivery-related matters with senior leadership and stakeholders, covering escalations, upsells, and ramp-downs
- Provide technical direction throughout the delivery process, keeping the architecture solid and forward-thinking while ensuring implementation meets quality expectations
- Write user stories and their corresponding acceptance criteria as part of an agile/scrum process
- Coordinate work across multiple disciplines and stakeholders to keep delivery progressing
- Ensure project delivery aligns with customer processes and methodologies, applying strong knowledge of different delivery models with an agile-focused approach
- Establish a continuous delivery risk management framework that enables proactive decision-making throughout the delivery lifecycle
- Monitor and enhance delivery productivity over time
- Serve as a technical resource for Data Engineers, helping ensure their work meets quality and timeliness standards
- Oversee production support and deployment responsibilities
Skills
- A minimum of 5 years of experience in a similar position
- At least one year of experience leading and managing teams
- Solid command of core Python for data engineering purposes
- Proficient in SQL for querying and processing data
- Hands-on experience with Apache Spark for large-scale data processing
- Familiarity with Anthropic Claude Code
- Experience using Databricks for data engineering and analytics tasks
- Excellent English proficiency (B2 level or higher)
- Experience with Apache Kafka for managing streaming data
- Working knowledge of Java
- Hands-on experience with PySpark
- Familiarity with Scala
- Experience with Spark Streaming
Benefits
- International projects with top brands
- Work with global teams of highly skilled, diverse peers
- Healthcare benefits
- Employee financial programs
- Paid time off and sick leave
- Upskilling, reskilling and certification courses
- Unlimited access to the LinkedIn Learning library and 22,000+ courses
- Global career opportunities
- Volunteer and community involvement opportunities
- EPAM Employee Groups
- Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn
Company Overview
Company H1B Sponsorship