Note: The job is a remote job and is open to candidates in USA. Zachary Piper Solutions is seeking a Data Cloud Engineer to support a federal modernization program focused on enterprise cloud data solutions and mission-critical analytics in secure DoD environments. The role will modernize enterprise data infrastructure by developing scalable ETL/ELT pipelines, cloud data platforms, governance controls, and reliable data access for analytics and operational decision-making.
Responsibilities
- Design, develop, and maintain production-grade ETL/ELT pipelines, with a strong emphasis on hands-on, high-performance PySpark development and Spark SQL optimization
- Build and maintain scalable data pipelines using Azure Synapse Analytics, Databricks, AWS Glue, and related cloud technologies
- Develop ingestion frameworks for structured and unstructured data, including relational databases, flat files, JSON, nested data, and REST APIs
- Implement full and incremental data loads, including CDC, late-arriving records, and rerunnable pipeline architectures
- Design and maintain cloud data lakes, warehouses, and analytics-ready datasets supporting enterprise reporting and operational decision-making
- Implement data quality and governance controls, including schema validation/enforcement, schema drift, RBAC, lineage, cataloging, and credential management
- Monitor and troubleshoot pipelines for performance, latency, failures, logging, alerting, and operational reliability
- Support CI/CD, automated deployments, rollback strategies, and environment promotions across cloud data environments
- Collaborate with cybersecurity, cloud, DevOps, and software engineering teams to support RMF, STIG, FedRAMP, and DoD security requirements, data migrations, integrations, and Agile delivery
Skills
- Requires an active Secret security clearance
- 10+ years of recent experience designing and operating scalable, production-grade data pipelines using Azure Synapse and/or Databricks, w/ a Bachelors degree in Computer Science, Data Science, Engineering, Information Systems, or a related field
- Strong hands-on PySpark development experience is required, including demonstrated high-performance coding skills developing ETL pipelines; candidates without this experience will not be considered
- Advanced experience with Spark SQL, Python, and SQL, including large-scale data transformation and optimization
- Experience ingesting structured and unstructured data and implementing full/incremental loads, CDC, and rerunnable pipeline architectures
- Strong experience with data quality, schema validation/enforcement, pipeline monitoring, logging, alerting, and troubleshooting
- Experience implementing CI/CD practices and automated deployments for data pipelines and cloud environments
- Experience with data governance, RBAC, and credential management in secure cloud environments
- Hands-on experience with Azure and/or AWS data services, including Azure Synapse, Azure Data Factory, Databricks, AWS Glue, Redshift, and S3
- Active Secret clearance
Benefits
- Medical
- Dental
- Vision
- 401k Plan
- PTO
- Holidays
- Sick Leave if required by law
- Fully remote position
Company Overview