Note: The job is a remote job and is open to candidates in USA. BigBear.ai provides AI-powered decision intelligence solutions for national security, supply chain management, and digital identity. The Data Engineer builds and maintains source adapters, normalization logic, and reliable ingestion and transformation pipelines that convert disparate data into a common risk-signal schema. The role also supports streaming data delivery, data quality, pipeline reliability, documentation, and collaboration with architecture and domain teams.
Responsibilities
- Build source adapters/connectors to ingest data from APIs, legacy systems, databases, and event streams
- Develop normalization and mapping logic to translate source-specific fields into the common risk-signal schema (including validation, enrichment, and standardization)
- Implement ETL/ELT pipelines with strong engineering rigor: testing, observability, error handling, retries, and backfills
- Produce and consume streaming events (e.g., Kafka topics) to support near-real-time signal delivery and downstream processing
- Partner with data architecture and domain SMEs to define and maintain data contracts, mappings, and lineage from source to normalized signal
- Ensure data quality and consistency (deduplication patterns, schema evolution handling, and reconciliation against source systems)
- Optimize pipeline performance and reliability (throughput, latency, and scalable processing patterns)
- Create and maintain technical documentation for adapters, transformations, and operational runbooks
- Some travel may be required within the DMV area
Skills
- Must maintain an active Top Secret security clearance
- Bachelor's Degree and 8 to 10 years of experience; Master's Degree and 6 to 8 years of experience
- 3–5 years of experience in data engineering, including building production-grade ingestion and transformation pipelines
- Strong experience with API integrations and ETL/ELT development in complex environments
- Experience integrating heterogeneous and/or legacy systems with inconsistent schemas and data quality
- Proficiency in Python or Java for building data services and transformation logic
- Solid SQL skills and working familiarity with NoSQL data stores
- Experience with REST/API frameworks and building maintainable, well-tested integration services
- Hands-on experience producing/consuming events in Kafka (producers/consumers) or an equivalent event streaming platform
- IC/DoD experience
- Python or Java
- REST/API frameworks
- Kafka producers/consumers
- SQL and NoSQL databases
Benefits
- This position is remote
Company Overview