Job Title: Databricks Architect
Location: United States
Work Mode: Remote
Job Type: Full-Time
Certification: Databricks Certification preferred
Job Summary
We are looking for an experienced Databricks Architect with strong expertise in the Databricks ecosystem, Medallion Architecture, cloud platforms (Azure/AWS), SQL, PySpark, and Python. The ideal candidate will be responsible for designing and implementing scalable, secure, and high-performance data architectures using Databricks, with hands-on experience in Auto Loader and modern data engineering practices.
Key Responsibilities
Design and implement scalable data lakehouse architectures using Databricks.
Develop and maintain data pipelines using PySpark, Python, and SQL.
Design and implement Medallion Architecture (Bronze, Silver, and Gold layers) for data ingestion, transformation, and consumption.
Implement incremental and near-real-time data ingestion pipelines using Databricks Auto Loader.
Integrate Databricks with Azure or AWS cloud services and data sources.
Optimize Spark workloads, SQL queries, cluster configurations, and data processing performance.
Implement Delta Lake solutions for reliable data storage, processing, and management.
Establish data governance, security, access controls, and best practices for enterprise data platforms.
Collaborate with data engineers, solution architects, and business stakeholders to translate requirements into technical solutions.
Provide technical leadership, architectural guidance, and support for production deployments.
Required Skills and Qualifications
Strong hands-on experience with Databricks architecture and the Lakehouse Platform.
Extensive experience designing and implementing Medallion Architecture.
Strong expertise in SQL, PySpark, and Python.
Hands-on experience with Databricks Auto Loader for scalable data ingestion.
Experience with either Microsoft Azure or AWS cloud services and their integration with Databricks.
Strong understanding of Apache Spark, Delta Lake, data modeling, and ETL/ELT pipelines.
Experience in performance tuning, troubleshooting, and optimizing large-scale data workloads.
Understanding of data governance, security, and production data pipeline best practices.
Ability to design scalable, reliable, and cost-effective data solutions.
Preferred Qualifications
Databricks Certified professional certification preferred.
Experience with Unity Catalog and data governance.
Knowledge of Delta Live Tables (DLT) / Lakeflow Declarative Pipelines.
Experience with CI/CD, Git, and infrastructure-as-code practices.
Experience with enterprise-scale data platform modernization and cloud migration.
Strong communication, problem-solving, and technical leadership skills.
Ideal Candidate Profile The ideal candidate should combine strong architectural expertise with hands-on implementation skills, particularly in Databricks, Medallion Architecture, Auto Loader, SQL, PySpark, Python, and Azure/AWS cloud environments.