Databricks Architect — Senior / Specialist
The Role
We're seeking a Databricks Architect (Senior/Specialist level) to serve as a senior technical authority for data architecture and analytics engineering on the Databricks Lakehouse platform across Cognify Analytics and our client engagements. In this role, you will design and own Databricks-based data platform architecture, lead complex data engineering initiatives, and ensure our data capabilities are enterprise-grade, governed, and future-ready. You will bridge data engineering, analytics, and AI/ML — combining deep hands-on expertise in Databricks, Python, and SQL with strong architectural judgment and stakeholder communication.
Must-Have Skills (Non-Negotiable)
Databricks — hands-on architecture and engineering experience on the Databricks Lakehouse Platform (production-grade, at scale)
Python — strong professional proficiency for data engineering and pipeline development
SQL — advanced proficiency for data modeling, transformation, and performance tuning
Candidates without demonstrable, hands-on experience in all three of the above will not be considered.
What You'll Do
Own the architecture and design of Databricks-based data platforms, including lakehouse design (Delta Lake), medallion architecture, and unified analytics layers.
Serve as the senior technical authority on Databricks platform design, providing guidance on architecture, tooling, modeling, and engineering standards across teams and projects.
Design and build ingestion, transformation, and orchestration pipelines using Python, SQL, PySpark, and Databricks Workflows.
Architect data mesh and data product strategies, defining domain ownership, data contracts, and self-service consumption patterns on Databricks.
Establish and evangelize best practices across the data lifecycle: ingestion, transformation, modeling, quality, observability, governance, and consumption.
Drive the integration of Databricks with AI/ML capabilities, including feature engineering pipelines, vector data infrastructure for RAG, and MLflow-based model workflows.
Lead complex data migration, platform modernization, and consolidation initiatives for enterprise clients across industries.
Evaluate emerging data technologies (Apache Iceberg, Unity Catalog, Delta Live Tables, Mosaic AI) to inform architectural decisions.
Solve complex, ambiguous, high-impact data architecture problems that span multiple teams, platforms, or organizational boundaries.
Drive cross-functional alignment between data engineering, analytics, AI/ML, platform engineering, security, and product teams.
Mentor senior data engineers and analysts, fostering a culture of technical excellence.
Represent Cognify Analytics in discussions with client stakeholders and leadership on data capabilities, strategy, and technical roadmaps.
Define and enforce data governance, compliance (GDPR, HIPAA, SOC2), and responsible data management standards across all Databricks environments.
Drive FinOps maturity for the Databricks platform, including compute optimization, cluster policies, storage lifecycle management, and cost forecasting.
What We're Looking For
Bachelor's, Master's, or equivalent professional experience in Computer Science, Data Science, Statistics, or a related field.
6–9 years of professional experience in data engineering, analytics engineering, or data architecture, with demonstrated technical leadership on at least a few large-scale engagements.
Mandatory, hands-on expertise in Databricks, Python, and SQL in production environments.
Strong working knowledge of the modern data stack: dbt, Airflow/Dagster, Fivetran/Airbyte, Spark, Kafka, and cloud-native data services.
Solid grasp of data modeling methodologies (Kimball, Data Vault, Activity Schema, OBT) and the judgment to apply them across contexts.
Experience with cloud data infrastructure on AWS, Azure, or GCP (any combination is acceptable, alongside Databricks).
Proven ability to influence technical direction and align technical and business stakeholders on complex architecture topics.
Strong understanding of data governance, data quality, cataloging, lineage, and regulatory compliance frameworks.
Experience mentoring engineers and contributing to a high-performing data team.
Strong understanding of how data platforms serve AI/ML workloads, including feature engineering, vector data, and model input/output pipelines.
Preferred Qualifications
Experience architecting Databricks-based platforms that directly serve LLM-based systems, RAG pipelines, and agentic AI architectures at enterprise scale.
Deep familiarity with lakehouse table formats: Delta Lake, Apache Iceberg, and Apache Hudi.
Track record of implementing data mesh, data product, or federated data governance patterns.
Experience with real-time analytics and streaming architectures: Kafka, Flink, Spark Structured Streaming.
Experience with advanced Databricks features: Unity Catalog, Delta Live Tables, Databricks Workflows, MLflow, and Mosaic AI.
Contributions to open-source data projects, data architecture publications, or industry standards bodies.
Background in financial services, healthcare, SaaS, or enterprise consulting requiring high data compliance, security, and operational rigor.
Experience leading data engineering across geographically distributed teams and multi-client engagements.
Why Join Cognify Analytics?
Join a team of industry veterans from Google, Meta, and top-tier tech companies.
Work on impactful, high-scale data and analytics projects with leading global clients.
Enjoy a flexible, remote-first culture focused on innovation and excellence.
Competitive salary, equity options, and continuous learning opportunities.
Shape the future of modern data platforms and AI-powered analytics at a rapidly growing company.
We are an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or any other protected characteristic.
Perks and Benefits
Unlimited PTO
Generous parental leave, above industry standards
Open communication with management and company leadership
Small, dynamic teams = massive impact
Medical, Dental, and Vision coverage
Access to Disability & Life insurance
Mental health and wellbeing support
Annual bonus program
Employer Stock Purchase Program (ESPP)
Yearly team building experiences
Mentorship and sponsorship opportunities
Manager resources and support
We are an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or any other protected characteristic.
Originally posted on