Note: The job is a remote job and is open to candidates in USA. Idexcel is hiring a Senior Data Engineer to develop and operate compliance data pipelines and products. The role transforms security and compliance telemetry into normalized data, monitoring logic, and actionable analytics, while supporting production releases and automation.
Responsibilities
- Build & Operate the Compliance Data Foundation: Serve as a principal individual contributor responsible for further development of newly acquired compliance and security datasets, building the end-to-end Databricks ETL pipelines that transform raw data into highly refined, high-utility data products designed to measure and monitor control performance and effectiveness
- Implement Medallion Data Pipelines: Design, optimize, and maintain robust ETL orchestration and automated pipelines to curate Bronze, Silver, and Gold data layers. Ensure raw, conformed, and consumption-ready compliance data products are highly structured and standardized
- Normalize Core Security & Compliance Data: Develop and implement data normalization schemas for diverse, high-volume telemetry feeds ingested from core corporate tools, including ServiceNow, Microsoft Defender, CrowdStrike, Zscaler, Wiz, Microsoft Entra ID, SailPoint, and many others
- Transform Telemetry to Monitored Logic: Partner with the data source and control domain owners to translate data sets into monitoring enablement and programmatic control-testing logic built directly into the data layer
- Produce High-Insight Data Products: Develop and maintain specialized compliance data products—including Databricks dashboards, reports, and analytic applications—that provide clear, actionable insights into compliance control performance, coverage gaps, and operational effectiveness enabling future agentic capabilities
- Support SDLC, Automation & Release Gates: Manage GitHub repositories, branching, and peer review standards. Establish automated job monitoring, exception handling, and code promotion processes to transition pipelines reliably into production
Skills
- Bachelor's degree with 5–8 years of experience spanning a hybrid of Cyber Security Operations (e.g., threat detection engineering, security telemetry analysis) and Security Compliance/GRC roles
- Advanced technical experience building ETL pipelines, orchestrating data workflows, and developing analytics in Databricks
- Proven capability to normalize, correlate, and test data structures sourced from security, identity, and governance platforms (ServiceNow, Defender, CrowdStrike, Zscaler, Wiz, Entra ID, and SailPoint)
- Solid understanding of security compliance frameworks, policy-to-control translation logic, and the deployment of continuous control monitoring (CCM) strategies within a secure cloud data lake
- Hands-on experience with modern software development workflows, including version control (Git/GitHub), peer review cycles, and release testing/approval gates
- Excellent verbal and written communication skills with a proven ability to bridge the gap between complex engineering requirements and business-facing risk management objectives
- Prior experience developing, deploying, or managing custom Databricks applications or Databricks Agents is a strong plus
- CISSP, CRISC, or Databricks Certified Data Engineer Associate/Professional
Benefits
- Remote work (Remote/Tampa, FL)
Company Overview
Company H1B Sponsorship