Note: The job is a remote job and is open to candidates in USA. NLB Services is recruiting for a Data Architecture & Engineering Manager; the hiring client is not identified. The role develops Databricks-based compliance and security data pipelines, normalizes telemetry, and delivers data products for monitoring control performance and effectiveness.
Responsibilities
- Build & Operate the Compliance Data Foundation: Serve as a principal individual contributor responsible for further development of newly acquired compliance and security datasets, building the end-to-end Databricks ETL pipelines that transform raw data into highly refined, high-utility data products designed to measure and monitor control performance and effectiveness
- Implement Medallion Data Pipelines: Design, optimize, and maintain robust ETL orchestration and automated pipelines to curate Bronze, Silver, and Gold data layers. Ensure raw, conformed, and consumption-ready compliance data products are highly structured and standardized
- Normalize Core Security & Compliance Data: Develop and implement data normalization schemas for diverse, high-volume telemetry feeds ingested from core corporate tools, including ServiceNow, Microsoft Defender, CrowdStrike, Zscaler, Wiz, Microsoft Entra ID, SailPoint, and many others
- Transform Telemetry to Monitored Logic: Partner with the data source and control domain owners to translate data sets into monitoring enablement and programmatic control-testing logic built directly into the data layer
- Produce High-Insight Data Products: Develop and maintain specialized compliance data products—including Databricks dashboards, reports, and analytic applications—that provide clear, actionable insights into compliance control performance, coverage gaps, and operational effectiveness enabling future agentic capabilities
- Support SDLC, Automation & Release Gates: Manage GitHub repositories, branching, and peer review standards. Establish automated job monitoring, exception handling, and code promotion processes to transition pipelines reliably into production
Skills
- Bachelor's degree with 5–8 years of experience spanning a hybrid of Cyber Security Operations (e.g., threat detection engineering, security telemetry analysis) and Security Compliance/GRC roles
- Advanced technical experience building ETL pipelines, orchestrating data workflows, and developing analytics in Databricks
- Proven capability to normalize, correlate, and test data structures sourced from security, identity, and governance platforms (ServiceNow, Defender, CrowdStrike, Zscaler, Wiz, Entra ID, and SailPoint)
- Solid understanding of security compliance frameworks, policy-to-control translation logic, and the deployment of continuous control monitoring (CCM) strategies within a secure cloud data lake
- Hands-on experience with modern software development workflows, including version control (Git/GitHub), peer review cycles, and release testing/approval gates
- Excellent verbal and written communication skills with a proven ability to bridge the gap between complex engineering requirements and business-facing risk management objectives
- CISSP, CRISC, or Databricks Certified Data Engineer Associate/Professional
- Applicants must be legally authorized to work in the United States
- Prior experience developing, deploying, or managing custom Databricks applications or Databricks Agents is a strong plus
Benefits
- Remote work arrangement
- Based on employment status and assignment, eligible employees may receive Medical, Dental and Vision Insurance, Health Insurance, Disability Insurance, etc. Eligibility and benefit terms vary by assignment.
Company Overview