Note: The job is a remote job and is open to candidates in USA. ECS is seeking a Senior Data Engineer to work remotely on advanced data integration and cloud solutions. The role leads the design, development, and optimization of scalable data pipelines and services supporting analytics, machine learning, predictive modeling, and real-time monitoring in secure cloud environments.
Responsibilities
- Develop production-grade ETL workflows using Python and Microsoft-based frameworks to ingest, transform, and validate large-scale structured and unstructured data
- Implement schema enforcement, data validation, and quality checks to maintain integrity across diverse sources
- Optimize pipelines for performance, scalability, and fault tolerance using open-source and cloud-native patterns
- Architect and manage Azure-based data solutions, including Data Lake Storage, Azure SQL, and cloud storage access from Python services
- Design and deploy workflow orchestration using Azure Data Factory or Foundry for scheduling, monitoring, and automation
- Ensure secure integration of APIs and services within the Microsoft ecosystem for seamless data exchange
- Build Python-based data services leveraging libraries such as Pandas, Pytorch, and other open-source frameworks for high-performance processing
- Implement logging, monitoring, and performance tuning for robust operational reliability
- Develop API endpoints and microservices to enable interoperability with analytics and ML platforms
- Work closely with data scientists, analysts, and cloud architects to deliver clean, reliable data for predictive modeling and real-time dashboards
- Apply data governance best practices, ensuring compliance, reproducibility, and auditability across workflows
- Contribute to Agile team processes, driving iterative improvements and shared problem-solving
Benefits
- Remote work arrangement
Company Overview
Company H1B Sponsorship