Note: The job is a remote job and is open to candidates in USA. Inceed has partnered with a company seeking a Senior Data Engineer to take ownership of a growing Databricks lakehouse platform. The role focuses on designing enterprise data architecture, building scalable data pipelines, modernizing legacy workloads, implementing governance, and delivering trusted data for reporting and AI use cases.
Responsibilities
- Own the technical design of the Databricks lakehouse platform
- Build and optimize pipelines using PySpark, Spark SQL, and Delta Live Tables
- Design enterprise data models and gold-layer reporting structures
- Support migration from legacy SQL Server, SSIS, Synapse, and ADF workloads
- Implement Unity Catalog governance and data access controls
- Tune performance, reliability, and cloud costs across the platform
- Establish CI/CD, testing, and environment promotion standards
- Partner with business users and BI teams on self-service analytics
- Review offshore partner work and maintain engineering quality
- Document architecture decisions and operational runbooks
Skills
- 8+ years of experience in data engineering, warehousing, or BI
- 5+ years building production platforms on Databricks or Spark
- Strong hands-on experience with PySpark and Spark SQL
- Experience owning the design of an enterprise data platform
- Advanced knowledge of Delta Lake and Databricks workflows
- Experience with dimensional modeling and star schema design
- Strong SQL skills, including complex stored procedure analysis
- Experience migrating legacy data platforms to modern cloud architecture
- Hands-on experience with Unity Catalog and data governance
- Ability to remain technical and hands-on in day-to-day development
- Our client unable to provide sponsorship at this time
Company Overview