Note: The job is a remote job and is open to candidates in USA. Kota is reimagining insurance and retirement benefits for the modern workforce through integrated and embedded benefits platforms. The Senior AI Engineer will help build Kota's generative AI capability by designing, shipping, and evaluating AI agent systems that improve how employers and employees use benefits data, while setting technical direction for AI architecture and practices.
Responsibilities
- Design and ship AI agent systems end to end, from early experiments through to production, owning the data access layer, retrieval strategy, and integration with Kota's existing infrastructure including Metabase and internal APIs
- Build and own the evaluation framework from scratch, including golden datasets, regression harnesses, and the ability to distinguish retrieval from generation failures; think adversarially about prompt injection, data boundary violations, and silent failure modes in a regulated context
- Set the technical direction for AI at Kota, with a seat at the table on agent architecture, and where AI creates the most leverage across the product
- Work across the pod and Platform team to surface new AI opportunities, collaborate with the AI Product Lead, and raise the bar on engineering practice through peer review and documentation
- Pick up slack and expand the role, this is an early pod with room to shape it; the remit will grow with you
Skills
- • 5+ years of software engineering experience, with at least 2 years working directly on LLM or AI agent systems in production
- • Strong grasp of agent architecture, including context management, tool use, retrieval strategies, and prompt design, and the ability to make and defend tradeoffs between them
- • Experience building evaluation frameworks, not just using them: golden datasets, regression testing, and retrieval vs. generation failure analysis
- • Comfort working across the stack
- • Clear written communication; you can explain a technical decision to a non-technical stakeholder without losing precision
- • Background in fintech, payroll, HR tech, or benefits; you understand why data freshness and auditability matter in regulated contexts
- • Experience with AWS, Azure, or the broader Microsoft ecosystem
- • Experience generating synthetic evaluation data to bootstrap eval coverage before real usage data is available
- • Experience working in early-stage or scale-up environments where the architecture is still forming
Benefits
- Health Insurance via Kota with generous allowance
- Workplace Pension with matched contributions up to 5% via Kota
- WFH stipend to support your home office
- 25 days of Paid Time Off, work hard and recharge fully
- Annual company-wide offsite and dedicated team offsite
- Attractive stock options for all employees
- Remote-first working style
Company Overview