Note: The job is a remote job and is open to candidates in USA. Dice is seeking an experienced Azure Data Fabric and ML Engineer to design and deploy advanced data pipelines and machine learning workflows for entity matching and deduplication across multiple data domains. The role involves building scalable MLOps solutions while collaborating with data scientists and analytics engineers to translate business requirements into production-ready ML solutions.
Responsibilities
- Build and optimize data pipelines and ML workflows within Microsoft Fabric for entity resolution, fuzzy matching, and data deduplication across CRM, EHR, finance, HR, and other data sources
- Design and operationalize MLOps pipelines encompassing model training, scoring, retraining, and deployment using SynapseML, PySpark, and Azure ML
- Develop reusable, modular components for identity resolution leveraging rule-based, fuzzy matching, and machine learning–based approaches
- Integrate and transform data from multiple heterogeneous sources, ensuring data quality and consistency across domains
- Collaborate with data scientists, analytics engineers, and subject matter experts to translate business requirements into production-ready ML solutions
- Apply data governance and privacy best practices throughout the pipeline lifecycle
- Optimize pipeline performance and resource utilization within Azure cloud infrastructure
Skills
- 10+ years of professional experience in data engineering, cloud platforms, or machine learning engineering
- Advanced proficiency with Microsoft Fabric, Azure Synapse, and Azure ML
- Strong hands-on expertise in Python and PySpark for data transformation and model development
- Demonstrated experience building and maintaining MLOps pipelines in production environments
- Solid understanding of entity resolution, fuzzy matching, and deduplication techniques
- Working knowledge of data governance, data privacy frameworks, and compliance best practices
- Experience integrating data from multiple enterprise systems and APIs in complex data ecosystems
Company Overview