Note: The job is a remote job and is open to candidates in USA. ECS is seeking a Senior Databricks Consultant to support its Professional Services and Managed Cybersecurity Services programs. The role consults with customers to design, optimize, and fine-tune Databricks environments for advanced analytics, machine learning, and AI-driven insights, while building scalable data pipelines and improving data infrastructure.
Responsibilities
- Design, build, and maintain scalable ETL/ELT pipelines in Databricks
- Optimize workflows and clusters for performance and cost efficiency
- Integrate Databricks with cloud platforms (AWS, Azure, or GCP)
- Implement data governance, security, and compliance standards
- Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality datasets and insights
- Troubleshoot issues and drive continuous improvement in data infrastructure
Skills
- Proven experience working with Databricks in production environments
- Strong proficiency in Python and/or SQL for data engineering
- Experience with Apache Spark for distributed data processing
- Knowledge of data lakehouse architecture and best practices
- Experience with cloud data platforms (AWS S3, Azure Data Lake, or GCP Storage)
- Hands-on experience building and managing ETL/ELT pipelines
- Strong understanding of data modeling, data quality, and performance tuning
- Familiarity with CI/CD pipelines and version control (Git)
- Ability to travel, as needed, to customer sites
- Must be a US Citizen per contract
- Bachelor's degree
- Will consider experience in lieu of a degree
- Preferably in Computer Science, Information Security, or a related field
Company Overview