Note: The job is a remote job and is open to candidates in USA. EPAM Systems is seeking a Senior Data Software Engineer to join its Data Engineering CoE. The role focuses on building and scaling Databricks data pipelines with PySpark and Python, supporting MLOps and machine learning teams, maintaining data quality, and optimizing large-scale data processing.
Responsibilities
- Support a team of data engineers to build pipelines used by MLOps and ML Engineers on ML modeling teams
- Develop, optimize, and maintain data transformation pipelines in Databricks (PySpark)
- Work with data stored in ADLS Gen2 and SAP HANA Data Lake, primarily in Delta/Parquet format
- Implement and maintain data quality checks, including schema validation, deduplication, enrichment, and tagging
- Communicate with stakeholders to understand business processes and model input data
- Tune performance for large-scale datasets
Skills
- - 3+ years of experience in data engineering or a related field
- - Proficiency in Python, PySpark, and Databricks, including Delta Lake
- - Familiarity with software version control tools such as GitHub and Git
- - Experience with CI/CD frameworks such as GitHub Actions
- - Knowledge of data lake technologies
- - Experience working with MS Azure
- - English proficiency at B2 level or higher
- - Familiarity with at least one other programming or scripting language, such as Java, SQL, or Scala
Benefits
- Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
- Opportunity to work abroad for up to two months per year
- Relocation opportunities within our offices in 55+ countries
- Corporate and social events
- Leadership development, career advising, soft skills and well-being programs
- Certifications, including GCP, Azure and AWS
- Unlimited access to EPAM's internal learning database
- Free English classes with certified teachers
- Discounts in local language schools, including online courses for the Kazakh language
- Participation in the Employee Stock Purchase Plan
- Monetary bonuses for engaging in the referral program
- Medical & family care package
- Six trust days per year (sick leave without a medical certificate)
- Coverage of psychology sessions of your choice
- Benefits package (sports activities, a variety of stores and services)
- Housing support program: preferential mortgage access via Otbasy Bank partnership
Company Overview
Company H1B Sponsorship