Note: The job is a remote job and is open to candidates in USA. AmbientSkye is hiring a Senior Data Engineer (Quantitative) to build and maintain data ingestion pipelines, datasets, and a knowledge graph with traceable provenance. The role also validates quantitative scoring and AI outputs across the platform’s life-sciences, regulatory, financial, industrial, and economic data applications.
Responsibilities
- Build source adapters and ingestion pipelines for public and customer-supplied data that are idempotent, efficient at bulk volume, and keep provenance on every record
- Assemble documented, cited datasets from heterogeneous sources: harmonise units, periods and categories; detect and flag conflicts rather than averaging them away; write the data dictionary
- Own the platform's existing pipelines for life-sciences, regulatory and financial filings data
- Extend the knowledge graph that links entities, evidence and assumptions, and keep it consistent as inputs change
- Calibrate and validate the quantitative scoring engine, which ranks candidate outcomes and measures how uncertain that ranking is: coverage testing, assumption sensitivity, and an honest write-up of limitations and failure cases, including when the engine is applied to a new problem domain
- Turn the platform's accuracy scorers and replay fixtures into a recorded evaluation suite that runs on a schedule
- Works most closely with the Director of Software Development on day-to-day priorities, data architecture, code review and delivery commitments
- Partners with the COO, who designed the scoring and calibration method, on validation work
- Works with the Senior Backend and Agent Engineer so the agents and scenario engines read data the way it was ingested, with provenance intact
- Writes dataset documentation and validation findings as deliverables in their own right, clear enough for non-engineering partners and customers
- Keeps customer-supplied data inside its tenant: row-level security on every table and query the role writes
- Maintains provenance and access controls on every ingested record, so any figure can be traced and any access explained
- Honours the licence and usage terms of third-party data sources
- Follows change management for schema changes: versioned migrations, reviewed pull requests and CI gates
- Emits audit events for state-changing data operations, and supports incident response for the pipelines the role owns
Skills
- 6+ years of data engineering in Python, with strong PostgreSQL: bulk loading (COPY), upserts, query tuning
- Working statistics: calibration, confidence or prediction sets, out-of-sample testing, small-sample pitfalls and leakage. Can tell when a result is too good to be true
- A record of building datasets from messy public sources with traceable provenance
- Hand-written SQL and comfort with row-level-security tenant isolation (no ORM)
- Clear technical writing
- US person (for example, a US citizen or permanent resident)
- Conformal prediction, Bayesian methods or forecasting evaluation
- Experience evaluating LLM output: labelled sets, precision and recall of extracted facts, LLM-as-judge
- Familiarity with economic, industrial, healthcare (NPPES, Open Payments, FDA) or SEC EDGAR data
- Knowledge-graph or entity-resolution experience; pgvector
Benefits
- Health Insurance
- Paid Leave
- Equity
Company Overview