Note: The job is a remote job and is open to candidates in USA. Shakti Solutions is seeking an AWS Databricks Consultant to support cloud-based data platform initiatives. The role focuses on designing and implementing Databricks environments and lifecycle management on AWS, establishing governance and CI/CD practices, and enabling AI/ML capabilities. The consultant will also translate business requirements into technical blueprints, conduct proof-of-concepts, and guide engineering teams on best practices.
Responsibilities
- Design and implement end-to-end lifecycle of Databricks platform on AWS
- Design and implement different environments like Dev/SIT/UAT/PROD for Databricks
- Expertise in AWS
- Implement data access controls, Lineage and auditing using Unity catalog
- Implement CI/CD pipelines between different environments
- Hands-on architecture experience within AWS, Azure, or GCP environments
- Deep expertise in Apache Spark, Delta Lake, SQL, and cluster resource management
- Translate business requirements into technical blueprints, run proof-of-concepts (POCs), and guide engineering teams on best practices
- AI/ML Enablement, experience in setting up AI and Genie for Databricks
- Experience with Delta lake house, Unity catalog, DBX notebooks, Python, Pyspark, Genie, Databricks Apps
Skills
- Expertise in AWS
- Hands-on architecture experience within AWS, Azure, or GCP environments
- Deep expertise in Apache Spark, Delta Lake, SQL, and cluster resource management
- AI/ML Enablement, experience in setting up AI and Genie for Databricks
- Experience with Delta lake house, Unity catalog, DBX notebooks, Python, Pyspark, Genie, Databricks Apps
Company Overview
Company H1B Sponsorship