Note: The job is a remote job and is open to candidates in USA. Resonate provides AI-powered consumer data, intelligence, and technology solutions for marketers and enterprise brands. The Data Engineer will design, build, maintain, and optimize large-scale data pipelines using Spark, Scala, and AWS, while supporting production operations and incorporating generative AI into engineering workflows.
Responsibilities
- Designing, developing, and maintaining the ETL/ELT pipelines that power the Resonate business - Spark and Scala on AWS EMR, working with terabyte-scale datasets across S3 and Snowflake
- Tuning multi-terabyte and petabyte scale Spark applications for performance and cost, and debugging the problems that only ever show up at that size
- Partnering with senior engineers and product management to take product requests from design and planning through to pipelines running in production
- Monitoring pipeline health in Grafana, tracking down data quality issues, and fixing the cause rather than the symptom
- Writing clean, testable code with thorough unit and integration tests, using Gen AI to move faster through the parts that used to slow you down
- Using Gen AI tools across pipeline development, monitoring, and day-to-day engineering work - and helping the team figure out where they genuinely help and where they do not
- Taking part in code reviews, technical design discussions, and sprint planning, with your point of view on architecture actually counting
- Supporting production operations, including an on-call rotation and incident response
Skills
- Around five years or more of professional experience in software engineering, data engineering, or a closely related field
- At least three years hands-on with Spark and Scala, specifically the DataFrame and Dataset APIs
- Proven experience tuning Spark applications at multi-terabyte or petabyte scale - you have done it, not just read about it
- Real debugging and problem-solving experience in the big data ecosystem, including the jobs that only fail in production
- Solid relational database experience
- Working knowledge of modern cloud stacks for processing big data - AWS (EMR, S3, Lambda), Kafka, Snowflake, Grafana, Hadoop, Elastic Stack, and Docker
- A strong grasp of the full software development lifecycle, from exploration and design through to delivery in production, plus good instincts on solution architecture, data structures, and data modeling
- Real enthusiasm for using generative AI to accelerate data engineering work - pipeline development, system monitoring, and developer productivity
- Hands-on experience building and maintaining big data pipelines with Gen AI
- Experience using probabilistic data structures
- A track record of implementing high cardinality data systems at scale
- A bachelor's degree in computer science, computer engineering, or equivalent experience
Benefits
- Uncapped growth potential
- A meaningful impact on the company
- The opportunity to work on cutting-edge, AI-powered marketing data and identity solutions
- A competitive 401(k) match
- An Open PTO policy that encourages real time off
- A remote-first environment with genuine flexibility and occasional travel for company and client events
- A flexible environment that empowers the team to collaborate seamlessly across different locations
- Tools and resources needed to succeed, whether working from home or in one of Resonate's offices
Company Overview
Company H1B Sponsorship