Note: The job is a remote job and is open to candidates in USA. ClearCaptions, LLC is dedicated to improving the lives of seniors and their caregivers through innovative communication technologies. The Sr. Data Engineer – Fabric will lead the design and implementation of enterprise-scale data solutions using Microsoft Fabric, collaborating with various teams to ensure reliable and scalable data operations while mentoring other data engineers.
Responsibilities
- Collaborates with cross-functional teams to understand data and analytical requirements and translates them into effective Microsoft Fabric–based data solutions
- Mentors and provides technical leadership to Data Engineers through code reviews, design reviews, knowledge-sharing sessions, and engineering best practices
- Establishes and maintains data engineering standards, reusable frameworks, development patterns, and operational best practices to improve consistency and scalability across the data platform
- Leads root cause analysis and resolution efforts for critical production issues impacting enterprise data pipelines, reporting solutions, and platform operations
- Reviews and provides guidance on solution designs, architecture decisions, and implementation approaches to ensure alignment with enterprise standards and best practices
- Designs, develops, and maintains end-to-end data ingestion and transformation processes using Microsoft Fabric OneLake, Lakehouses, Data Pipelines, Dataflows Gen2, and Notebooks
- Formats, cleanses, and stores data in a structured manner to ensure data quality and accessibility for reporting and analysis, following Medallion Architecture
- Works closely with reporting and analytics teams to ensure datasets are accurate, well-modeled, and optimized for Power BI and other analytical workloads
- Creates comprehensive documentation for data pipelines, processes, and solutions
- Participates in data governance activities to ensure data integrity, security, and compliance
- Utilizes source control to manage and track changes to data pipelines and codebase
- Provides level II/III support to troubleshoot and resolve data-related issues and inquiries
- Provides ancillary support for Machine Learning (ML) and Artificial Intelligence (AI) processes
- Evaluates emerging technologies, industry trends, and architectural patterns related to data engineering, analytics, artificial intelligence, and cloud platforms and provides recommendations for adoption
- Manages and optimizes Microsoft Fabric platform performance by monitoring capacity utilization, query performance, Lakehouse and Warehouse workloads, Delta table maintenance (OPTIMIZE/VACUUM), data partitioning, storage consumption, and pipeline execution to ensure scalable, reliable, and cost-effective data operations
- Develops and maintains monitoring, alerting, and observability solutions for Microsoft Fabric pipelines, notebooks, Lakehouses, Warehouses, semantic models, and related data platform components to ensure operational reliability and rapid issue resolution
- Performs other duties as assigned
Skills
- Bachelor's degree in computer science, information technology, or a related field (or equivalent experience)
- A minimum of six (6) years of proven experience in data engineering, ETL development, and data warehousing
- Demonstrated experience leading complex data engineering projects from architecture and design through implementation and operational support
- Demonstrated ability to lead through influence, collaboration, and technical expertise, including mentoring Data Engineers and cross-functional technical teams, leading initiatives and workstreams, and driving successful cross-functional outcomes
- Experience establishing engineering standards, governance processes, development frameworks, and operational best practices
- Advanced troubleshooting and performance optimization experience across cloud data platforms, distributed processing frameworks, and enterprise data solutions
- Strong hands-on experience with Microsoft Fabric, including OneLake, Lakehouses, Data Pipelines, Dataflows Gen2, and/or Notebooks
- Solid understanding of Medallion Architecture, Delta Lake concepts, data modeling, and modern analytics patterns
- Experience with data governance, data quality, and data security frameworks and standards
- Solid grasp of Continuous Integration Continuous Delivery (“CI/CD”), including source control, deployment pipelines, and release management practices
- Experience with Git-based source control and collaborative development workflows
- Expertise in integrating Fabric solutions with Power BI, Azure services, and external data sources
- Experience monitoring and tuning cloud-based data platforms for performance, scalability, and cost optimization
- Experience with Apache Spark and PySpark within Microsoft Fabric
- Excellent verbal and written communication skills, presentation, problem-solving skills and attention to detail
- Self-starter with strong organizational and time management skills, self-directed and able to handle multiple priorities with demanding timeframes
- Ability to work independently and collaboratively with colleagues and staff to create a high-quality, results-driven, team-oriented, fast-paced environment
- Demonstrated ability to use discretion, make sound decisions, and maintain confidentiality
- Willingness and ability to work flexible hours
- Proficiency in MS Office (including Excel, Outlook, PowerPoint, Visio, & Word) and modern communication tools for virtual teams (e.g., MS Teams, Slack)
- Expertise in developing data pipelines to support Machine Learning (ML) and / or Artificial Intelligence (AI) tools is a plus
Company Overview