Note: The job is a remote job and is open to candidates in USA. Cardlytics is an industry-leading purchase intelligence and incentives platform founded in 2008. They are seeking a Principal Software Engineer, Tech Lead - Platform to lead the evolution of their data compute environment on AWS, making architectural and operational decisions that enhance data accessibility and usability for analysts, engineers, and business stakeholders.
Responsibilities
- Design and evolve the Cardlytics next-generation architecture; establish reference architectures and standards for batch, streaming, ML, and AI/agentic workloads
- Build out the semantic layer, feature store, and data ontology that make Cardlytics data consumable by AI/agentic applications; enable governed, self-service access for both human and machine (LLM/agent) consumers
- Serve as the primary technical partner for analysts, engineers, and business users; develop self-service guides, onboarding materials, and best-practice documentation to reduce friction across all user groups
- Set technical direction for the platform and mentor engineers across the data and platform teams; drive engineering excellence and best practices, and manage the vendor relationship
- Define monitoring standards and SLAs, lead performance tuning for Spark jobs, open table formats (e.g. Delta, Hudi), and SQL query engines; own incident response and root-cause analysis
- Drive adoption of a unified governance layer (e.g., Databricks Unity Catalog) for centralized RBAC, column-level security, data lineage, and metadata management; implement and maintain security best practices including encryption and audit logging
- Define the operating model and guardrails for the managed lakehouse platforms (including Databricks on AWS) — compute policies, access model, catalog structure, and cost governance — with day-to-day administration executed by the platform and domain teams working within those standards
- Own the CI/CD strategy for lakehouse and data platform deployments (Terraform, GitHub Actions, and platform-native packaging such as Asset Bundles); enforce DevOps practices across engineering teams
Skills
- 7+ years of experience in data engineering or data platform roles, with at least 2 years in a technical lead or architecture capacity
- Deep, hands-on expertise with the leading industry platforms, frameworks, and tools: Databricks, AWS EMR or similar, Delta Lake, Unity Catalog, Spark (PySpark), and SQL; with a solid understanding of compute/cluster management on AWS
- Proven experience designing and delivering Lakehouse architectures and large-scale ELT/ETL pipelines in production environments
- Proficiency in Infrastructure as Code tools, such as Terraform
- Demonstrated experience supporting diverse stakeholder groups including analysts, engineers (including ML engineers), and non-technical business users
- Strong understanding of data governance, data modeling, and cloud security best practices
- Excellent communication and collaboration skills, with the ability to translate technical concepts for non-technical audiences
- Experience designing or operating a semantic layer, feature store, or data ontology / knowledge graph in a production environment
- Familiarity with managing data for ML systems and AI/agentic applications, such as training pipelines or inference for LLM-powered and agentic tools
Benefits
- Flexible paid time off plus company holidays
- Medical, dental, and vision insurance begins on your first day
- 401(k) retirement plan with company match, plan also includes a student loan debt repayment option
- Employee Stock Purchase Plan
- Educational assistance for continuing education
- Lifestyle Spending Account for physical, emotional, and financial wellness (like gym memberships, home down payments, art classes, park passes, and more!)
- Complimentary Calm app subscriptions to support employee mental health and wellbeing
Company Overview
Company H1B Sponsorship