[Remote] Senior Databricks Engineer / Architect

Note: The job is a remote job and is open to candidates in USA. Connect Tech+Talent is seeking a Senior Databricks Engineer / Architect to support an enterprise data platform for one of its clients. The role focuses on designing, building, and optimizing Databricks-based Data Lakehouse solutions, automating infrastructure with Terraform, developing scalable data pipelines, and implementing governance, security, and CI/CD practices.

Responsibilities

• Databricks Architecture & Engineering: Architect, build, and maintain production-grade data pipelines using Spark, Delta Lake, Databricks Workflows, and Unity Catalog
• Infrastructure as Code (IaC): Automate and manage Databricks workspace configurations, clusters, security, and cloud resource provisioning using Terraform
• Data Pipeline Development: Design scalable ETL/ELT batch and streaming data processing pipelines using Python, PySpark, and SQL
• Performance Optimization: Troubleshoot and tune Spark jobs, memory allocation, and query execution for high-volume enterprise workloads
• Data Governance & Security: Implement Unity Catalog for access control, lineage, and data governance across multi-environment setups
• CI/CD & MLOps Integration: Establish and maintain automated CI/CD deployment pipelines (GitHub Actions, Azure DevOps, or AWS CodePipeline) for Databricks assets and Terraform configurations
• Collaboration: Work closely with data architects, software engineers, and analytics teams to translate business requirements into efficient technical architecture

Skills

• 10+ years of hands-on experience in building enterprise data solutions on the Databricks platform (Delta Lake, PySpark, Unity Catalog, Databricks SQL)
• Mandatory experience writing, maintaining, and deploying Terraform modules specifically for Databricks resources and underlying cloud infrastructure
• Solid hands-on experience in either Azure (Azure Databricks, ADF, ADLS Gen2) or AWS (S3, IAM, EMR)
• Mastery of Python, PySpark, and advanced SQL
• Strong background in data modeling, lakehouse architecture, real-time streaming, and batch ingestion
• Experience with Git version control and CI/CD tools for automated code/infrastructure deployments

Benefits

• 100% Remote
• Long-term Contract

Company Overview

Connect Tech+Talent is a Global Technology Partner which brings together highly qualified technology professionals with great companies. It was founded in 1996, and is headquartered in Austin, Texas, USA, with a workforce of 11-50 employees. Its website is https://www.connecttechtalent.com/.

Company H1B Sponsorship

Connect Tech+Talent has a track record of offering H1B sponsorships, with 2 in 2025, 2 in 2023, 1 in 2020. Please note that this does not guarantee sponsorship for this specific role.