Note: The job is a remote job and is open to candidates in USA. Five Rivers Analytics, an Akima company, supports federal IT and the Marine Corps through the Logistics Analytics Data Services program. The Data Engineer will build and maintain scalable Databricks data pipelines, implement CDC and SCD architectures, develop Lakehouse data models, and ensure secure, auditable, high-performance data products. The role also focuses on data governance, quality monitoring, performance tuning, and collaboration with downstream data consumers.
Responsibilities
- Pipeline Orchestration: Design, build, and maintain scalable batch and streaming ETL/ELT pipelines using Python, SQL, and built-in Databricks Workflows
- Change Data Capture (CDC): Establish efficient pipelines to ingest incremental source changes (inserts, updates, and deletes) into the lakehouse using native Databricks capabilities
- Lakehouse Architecture: Implement and manage data layouts using the Medallion Architecture (Bronze, Silver, Gold layers) to clean, enrich, and optimize structured and semi-structured datasets
- Data Governance & Security: Administer schemas, catalogs, and secure access permissions using Unity Catalog to ensure strict data governance
- Performance Tuning: Optimize SQL queries, caching, indexing, and cluster configurations to reduce execution times and platform compute costs
- SCD & History Implementation: Design and implement robust Slowly Changing Dimension (SCD) patterns and historical truth tables in Delta Lake
- Requirements & Collaboration: Work closely with dashboard developers, data analysts, and data scientists to deeply understand and capture their data requirements
- Source System Mastery: Proactively investigate and learn about various source systems to understand how they work and how data is generated and extracted
- Data Modeling: Create clean, optimized, and reusable semantic data models in Databricks SQL that serve as the "single source of truth" for the business
- CI/CD & Git Integration: Implement software development best practices using Git integration in Databricks Repos for version control, code review, and automated deployments
- Data Quality Monitoring: Implement automated data validation, logging, and alerting systems using native Databricks data quality tools to ensure high data integrity
Skills
- Current and active Secret security clearance
- Bachelor's degree in CS, IT, Engineering, Math, or related field, or equivalent experience
- 8+ years' experience (Bachelor's) or 6+ years (Master's)
- Strong understanding of relational database principles, indexing, transaction management, and query optimization
- Hands-on experience with SQL
- Knowledge of database security controls and encryption
- Experience in Agile/SAFe environments
- Experience modeling data for use in Foundry (Ontologies)
Benefits
- Medical insurance
- Dental insurance
- Vision insurance
- Life insurance
- 401(k)
- A range of other voluntary benefits
- Paid Time Off (PTO) for regular full-time and part-time employees
- Remote work arrangement
- Growth opportunities
- Excellent retirement options
Company Overview