Note: The job is a remote job and is open to candidates in USA. Source Select Group is seeking a Sr. Lead AI & Azure Data Engineer to lead a team in the design, development, and delivery of machine learning models and data solutions on Microsoft Azure and Fabric. The role involves architecting data pipelines, collaborating with stakeholders, and mentoring team members to ensure high-quality data delivery and adherence to best practices in cloud data engineering.
Responsibilities
- Lead Azure Data Engineer/Architect responsible for designing, building, and optimizing data solutions on Microsoft Azure
- Collaborate closely with stakeholders to gather requirements, translate them into technical designs, and ensure data quality, security, and governance standards are met
- Day-to-day responsibilities include hands-on development, performance tuning, code reviews, and mentoring other data engineers
- Architect and build scalable pipelines using Databricks, Apache Airflow, Fabric Data Factory, Azure Data Engineering, and Microsoft Fabric (Lakehouse, OneLake, Semantic Models)
- Implement medallion architecture and modern data warehousing
- Ensure scalability, performance, and resilience
- Own full lifecycle: feature engineering → model design → training → validation → deployment → monitoring
- Drive model selection, tuning, and evaluation strategies
- Deliver predictive analytics tied to project and financial data
Skills
- 7+ years in Azure Data Engineering
- 3+ years of work experience with Azure Data Factory
- 1+ years of work or strong knowledge of Azure Service Fabric
- 4+ years of work experience with Azure Databricks
- 3+ years in technical leadership roles
- Bachelor's degree required
- Expert in Python
- Advanced T-SQL
- Strong Spark/PySpark
- Deep experience in Azure Data Factory (ADF) for ML pipelines
- Data engineering – ETL/ELT, data warehousing, medallion architecture
- Advanced experience with Azure ecosystem
- Databricks
- Microsoft Fabric
- Apache Airflow
Benefits
- Remote USA- in approved states
- Must be willing to travel onsite for final interview and if selected for orientation and yearly company meeting
- Full-time, remote position
Company Overview