Note: The job is a remote job and is open to candidates in USA. QualityAI is an AI-powered Quality Engineering Company seeking a Data Engineer to join its growing team. The role focuses on building and optimizing scalable batch and streaming data pipelines, using Python, PySpark, Databricks, Snowflake, and modern cloud data platforms. The Data Engineer will also support orchestration, monitoring, CI/CD, cloud optimization, and collaboration across business, analytics, and engineering teams.
Responsibilities
- Strong hands-on experience in Python and PySpark, with a solid understanding of Spark concepts, performance tuning, and scalable data processing
- Experience working with modern data management platforms such as Snowflake, Databricks, or similar cloud-based data platforms
- Good understanding of data pipeline design, development, orchestration, and monitoring, including batch and streaming data processing
- Familiarity with Medallion Architecture concepts, including bronze, silver, and gold layer design and best practices for data transformation
- Strong understanding of autoscaling, cluster optimization, and cost-efficient data processing in cloud or distributed data environments
- Experience with CI/CD, DevOps practices, and version control, with preference for candidates who have exposure to Terraform or infrastructure-as-code tools
- Ability to collaborate with business, analytics, and engineering teams to design reliable, reusable, and scalable data solutions
Skills
- 1. Experience building end-to-end data pipelines (batch/streaming, orchestration, monitoring)
- 2. Practical exposure to modern cloud data platforms (Databricks, Snowflake)
- 3. Strong hands-on PySpark with Spark performance tuning (joins, partitioning, caching, optimization)
- -Strong hands-on experience in Python and PySpark, with a solid understanding of Spark concepts, performance tuning, and scalable data processing
- -Experience working with modern data management platforms such as Snowflake, Databricks, or similar cloud-based data platforms
- -Good understanding of data pipeline design, development, orchestration, and monitoring, including batch and streaming data processing
- -Familiarity with Medallion Architecture concepts, including bronze, silver, and gold layer design and best practices for data transformation
- -Strong understanding of autoscaling, cluster optimization, and cost-efficient data processing in cloud or distributed data environments
- -Ability to collaborate with business, analytics, and engineering teams to design reliable, reusable, and scalable data solutions
- -Experience with CI/CD, DevOps practices, and version control, with preference for candidates who have exposure to Terraform or infrastructure-as-code tools
Benefits
- Internal rotation and international mobility opportunities to grow your career.
- Opportunity to grow with the company.
- Flexible and casual culture.
- Employee events and access to amenities and games provided from one of our Employees Centers.
- QualityAI will match your 401k contributions.
- Competitive healthcare benefits.
- QualityAI will match towards your HSA if you choose to participate.
- QCraft Learning & Development platform with 50,000+ courses, 300+ virtual labs, mentorship and leadership programs, professional tribes, sponsored certifications, and more.
- Bonuses via the Client Referral and Employee Referral Programs.
- Bonuses and spot awards through the Qudos platform for celebrating employee achievements.
- QualityAI Employee Perks with discounts on travel, electronics, and more.
Company Overview
Company H1B Sponsorship