Note: The job is a remote job and is open to candidates in USA. Salas O’Brien is an employee-owned engineering and professional services firm providing integrated engineering and consulting services across numerous industries. The Databricks Platform Engineer / Architect will own the Databricks platform end to end, administering and architecting its environments while advising teams on scalable, secure, governed, and high-performing solutions. The role also oversees infrastructure automation, cost governance, identity and access management, networking, security, and adoption of emerging Databricks capabilities.
Responsibilities
- Administer the Databricks platform across multiple environments, including workspaces, Unity Catalog, cluster policies, budget policies, and the guardrails that enable teams to move quickly while maintaining consistency and control
- Manage identity and access, including Microsoft Entra ID single sign-on, account-level SCIM group provisioning, service principals, and scalable access-control models
- Partner with IT Infrastructure and Security to manage the platform’s networking and security posture, including customer-managed virtual private clouds, private connectivity, encryption, egress controls, audit logs, and system tables connected to enterprise monitoring tools
- Manage infrastructure and workloads through code using Terraform, Databricks Asset Bundles, and CI/CD practices, with no manual production configuration whenever automation is practical
- Establish and maintain cost-governance practices, including tagging standards, business-unit showback, budget controls, and recommendations for reducing unnecessary platform spending
- Serve as a trusted architecture advisor to data scientists, data engineers, analysts, and business stakeholders before solutions are designed and deployed
- Provide current, practical, and technically sound recommendations that balance organizational standards with business needs
- Evaluate emerging platform capabilities and determine how they should be adopted within the organization
Skills
- Deep, current, production-level Databricks administration experience on AWS, or significant Azure Databricks experience with a credible path to supporting AWS
- Experience designing and managing Unity Catalog in a multi-workspace environment
- Demonstrated experience developing catalog structures and access-control models that support a growing organization
- Experience integrating Databricks with Microsoft Entra ID, including single sign-on and SCIM provisioning
- Strong Terraform and CI/CD experience for infrastructure and platform management
- Working knowledge of cloud networking concepts, including virtual private clouds, private connectivity, encryption, egress controls, and related security-review processes
- Ability to build a strong partnership with IT Infrastructure and Security teams, clearly communicate platform needs, follow enterprise standards, and advocate constructively for architecture improvements
- Sound judgment in determining when the platform's default approach is appropriate and when organizational requirements call for a different solution
- Strong written and verbal communication skills
- Experience supporting government, military, or controlled-data environments, including Controlled Unclassified Information and Cybersecurity Maturity Model Certification requirements
- Experience with enclave separation, controlled environments, and regulated-data operations
Benefits
- Performance-based bonuses
- Equity participation
- Medical, dental, and vision insurance
- 401(k) with company match
- Paid time off and company holidays
- Wellness programs and employee-assistance resources
- Professional development support
- Remote work arrangement
Company Overview