Note: The job is a remote job and is open to candidates in USA. TheFutureSociety operates an AI astrology platform that combines deterministic calculations, retrieval, and large language model generation. The Senior AI / Retrieval Engineer will harden the production retrieval layer by implementing deterministic lookups, improving loader reliability, enforcing CI validation, adding retrieval telemetry, and achieving the required evaluation accuracy.
Responsibilities
- Deterministic keyed lookup. Our calculator already knows the correct answer — which planet, which house, which aspect. Today that fact is handed to an embedding search that sometimes returns the wrong one. Replace it: the calculator emits an exact key, the system looks it up directly, and an unmatched key returns nothing rather than a nearest neighbour. Spec and key taxonomy provided
- Loader hardening. Strict filename and key resolution. Fail fast on a missing or malformed file, support prefix-matched keys, surface errors instead of degrading quietly
- Validation gate in CI. We have an audit script that scores content rows against our editorial standard. Wire retrieval to honour its exclusion list and make the gate block a failing row from shipping
- Retrieval telemetry that distinguishes a miss from a logging gap. Some paths currently log a null result whether or not retrieval succeeded. We need to be able to tell the difference from the dashboard
- Prove it. We supply a 100-question evaluation set. After your changes, the system must answer at decision-support accuracy ≥ 60%, measured against our Swiss Ephemeris engine as ground truth. We QA the results — you do not grade your own work. Engine access provided so you can verify before submitting
Skills
- Strong Python. Production RAG experience that is yours, not a tutorial
- Postgres and pgvector, hybrid search, reranking, and the judgement to know when retrieval is the wrong tool
- Real evaluation discipline — you have built an eval set and used it to reject your own work
- Version control hygiene: branches, pull requests, reviewed merges. No manual changes to running services, ever
- Bonus, not required: Supabase, Next.js, SSE streaming
Benefits
- Paid 48-hour trial — $1,000 flat, paid on delivery whether or not we continue.
- $300 completion bonus if all five are delivered and demonstrated within 24 hours, on the same standard as the fee: everything working, in a reviewed pull request, with the 100-question run at or above 60%.
- Remote.
- Ongoing month-to-month, scoped and priced per milestone.
- You work directly with the owner.
- Decisions same day, no committee.
- Engine access provided so you can verify before submitting.
- We supply the key taxonomy, the audit script, the editorial standard, the evaluation set, engine access, repo and staging access, and direct access to the owner.
Company Overview