Senior/Lead Data Engineer (Databricks)
Experience: 8-14 Years
Location: Remote
Job Summary
We are looking for an experienced Databricks and Data Mesh Technical Lead to provide technical leadership across the Databricks and Data Mesh ecosystem. The role is responsible for translating business and Data Product requirements into scalable technical solutions, driving architecture and delivery, strengthening Data Mesh governance, and ensuring platform security, performance, cost optimization, and operational excellence. The ideal candidate should have strong hands-on experience with Databricks, Data Mesh, Unity
Catalog, data governance, cloud data platforms, and technical leadership.
Key Responsibilities
Provide technical leadership across the Databricks and Data Mesh ecosystem.
Translate business and Data Product requirements into scalable solutions and delivery roadmaps.
Own the technical backlog, architecture, delivery, enhancements, defects, and technical debt.
Lead the development and improvement of Data Mesh governance and self-service tools.
Ensure compliance with Unity Catalog, security, least-privilege access, governance, and audit requirements.
Own Usage and Cost Dashboards, ensuring accurate and actionable insights.
Communicate technical options, dependencies, risks, and constraints to stakeholders.
Support production operations, escalated incidents, and service improvements.
Lead engineers through technical guidance, performance feedback, and best practices.
Manage project scope, resources, schedules, budgets, and delivery.
Use platform metrics, cost insights, and stakeholder feedback to continuously improve Data Mesh capabilities.
Must-Have Skills
Strong software and solution-engineering experience in cloud data-platform applications, APIs, automation, and self-service capabilities.
Experience with technical architecture and scalable application delivery.
Working knowledge of Databricks and modern data-engineering practices.
Understanding of Data Mesh, Unity Catalog, data governance, RBAC, ABAC, security, and auditability.
Experience with cloud platforms, DevOps, monitoring, and production support.
Strong technical leadership, problem-solving, stakeholder-management, and communication skills.
Experience leading engineering teams and managing multiple projects.
Knowledge of Agile and Waterfall delivery methodologies.
Preferred Skills
Databricks Data + AI Platform Administration
Terraform or Infrastructure as Code
Analytics and AI/ML support
LLM and Generative AI/ Agentic AI knowledge
OpenAI Codex or Anthropic Claude Code
Certifications
AWS Platform Architect
Azure Platform Architect
Databricks Data Engineer Professional