Note: The job is a remote job and is open to candidates in USA. Reqroute, Inc. is recruiting for a Big Data Engineer to support a digital transformation and data engineering initiative in the chemical/manufacturing industry. The role will design and build production-grade Azure and Databricks data pipelines integrating enterprise applications, plant operational technology, and industrial data into a governed unified data platform. The engineer will also implement data quality, security, monitoring, governance, CI/CD, and performance optimization practices.
Responsibilities
- Design and build scalable batch, CDC, near-real-time, and streaming data pipelines
- Build pipelines using Azure Databricks, Azure Data Factory, PySpark/Spark, SQL, ADLS/OneLake, and related Azure services
- Integrate data from ERP, CRM, SaaS, SAP S/4HANA/DataSphere, APIs, manufacturing applications, and OT systems
- Ingest and contextualize plant/industrial data from Historian, SCADA, PLC, IoT/IIoT, telemetry, and time-series environments
- Work with industrial connectivity/protocol concepts such as OPC UA and MQTT, along with secure OT-to-IT integration patterns
- Develop landing → curated → semantic/consumption layers within the Unified Data Platform
- Implement data contracts, schema/versioning, SCD handling, partitioning, caching, clustering, and performance optimization
- Build dimensional and semantic models supporting Power BI datasets, APIs, analytics, AI/ML applications, and agents
- Partner with plant controls/OT teams around signal quality, security, change control, network boundaries, and downtime windows
- Implement data-quality checks for freshness, completeness, schema changes, drift, and validation
- Establish monitoring, lineage, alerting, troubleshooting procedures, and production runbooks
- Implement RBAC, Key Vault/secrets management, data classifications, retention, governance, and MDM standards
- Automate testing and deployment through Git-based CI/CD and structured dev/test/prod environments
- Monitor and optimize Databricks/Azure performance and cloud costs
- Create documentation, data dictionaries, technical specifications, runbooks, and knowledge-transfer materials
Skills
- • 5+ years of production data engineering experience building enterprise-scale data pipelines
- • Strong hands-on experience with Azure Databricks
- • Strong hands-on experience with Azure Data Factory (ADF)
- • Advanced PySpark/Spark, Python, and SQL skills
- • Experience with Delta Lake/Lakehouse or similar modern data architectures
- • Experience developing batch, CDC, and/or streaming pipelines
- • Strong understanding of Spark Structured Streaming and data-processing performance optimization
- • Experience integrating multiple enterprise source systems through APIs, databases, files, events, and streaming technologies
- • Experience with Azure technologies such as ADLS, Synapse/Fabric, Azure SQL, Event Hubs, Key Vault, Azure DevOps , or equivalent services
- • Experience implementing data quality, testing, monitoring, observability, governance, and CI/CD
- • Strong communication skills with the ability to independently work with technical and nontechnical stakeholders
- • Critical OT / Manufacturing Requirement
- • Candidates need meaningful experience with OT, IoT, industrial streaming, or time-series data
- The ideal candidate has worked in continuous or batch manufacturing or an adjacent industrial environment such as chemicals, pharmaceuticals, CPG, oil & gas, energy, or utilities
Benefits
- REMOTE ROLE
- 6+ Months Contract (CTH)
Company Overview