Note: The job is a remote job and is open to candidates in USA. Cardlytics is an industry-leading purchase intelligence and incentives platform that helps businesses attract, understand, and incentivize consumers through digital reward programs. The Senior Principal Data Engineer will architect and lead an MLOps and forecasting platform, building scalable Databricks-based data infrastructure, automated pipelines, model deployment, and forecasting services while mentoring engineers and aligning technical strategy with business goals.
Responsibilities
- Design and architect a unified MLOps and forecasting platform on Databricks, leveraging MLflow for model orchestration and Databricks SQL for high-scale data processing
- Build and maintain automated pipelines to handle critical business use cases, specifically:
- Developing models that simulate past campaign performance using historical transaction data to forecast future outcomes
- Real-time monitoring and prediction of budget utilization to prevent over/under delivery
- Implementing robust Incrementality Testing framework to quantify the causal impact of advertising exposure
- Establish CI/CD and deployment standards to ensure models are scalable, frequently refreshed, and easily debugged
- Define "clean data" and telemetry-first standards. Analyze large datasets to identify trends and insights that improve model performance and advertising outcomes
- Partner with Applied Scientists to explore GenAI patterns and automated architectures to enhance team workflows
- Collaborate with product and business stakeholders to translate high-level business goals into technical roadmaps that prioritize explainable, scalable, and accurate engineering solutions
- Act as a technical mentor for senior and staff engineers, fostering a culture of operational excellence and rapid dev velocity
Skills
- * Bachelor's or Master's degree in Computer Science, Engineering, or a related field; PhD preferred
- * 10+ years of experience in data engineering, with a significant focus on building and scaling MLOps pipelines in a production environment
- * Expertise in the Databricks Data Intelligence Platform, including Unity Catalog and MLflow, for big data and ML workloads
- * Proficiency in Python, SQL, and PySpark, with deep knowledge of data modeling and semantic layer design for large-scale analytics
- * A proven track record of leading large-scale architectural migrations and managing technical debt
- * Strong understanding of time-series forecasting, statistical modeling, or causal inference (e.g., Lift Analysis)
- * Exceptional communication skills, with the ability to influence executive stakeholders and align technical strategy with business goals
- Familiarity with or willingness to work within this environment is required
- * Bachelor's or Master's degree in Computer Science, Engineering, or a related field; PhD preferred
- * Expertise in integrating complex prediction models (ranking, retrieval, and conversion) into high-traffic Marketplace funnels
- * Experience with real-time systems and large-scale data processing technologies (e.g., Kafka, Kinesis)
- * Familiarity with Learning to Rank (LTR) models and their productionization challenges
- * Demonstrated ability to implement AI-powered assistants for product development, alert triage, and workload migration
Benefits
- Flexible paid time off plus company holidays
- Medical, dental, and vision insurance begins on your first day
- 401(k) retirement plan with company match, plan also includes a student loan debt repayment option
- Employee Stock Purchase Plan
- Educational assistance for continuing education
- Lifestyle Spending Account for physical, emotional, and financial wellness (like gym memberships, home down payments, art classes, park passes, and more!)
- Complimentary Calm app subscriptions to support employee mental health and wellbeing
Company Overview
Company H1B Sponsorship