Job Description:
• Define end-to-end requirements for Artificial Intelligence (AI) capabilities, from model behavior to system behavior to user experience
• Translate model capabilities, evaluation results, and technical constraints into clear product and system decisions
• Make trade-offs across quality, latency, cost, reliability, safety, and UX
• Work closely with Machine Learning (ML), backend, and client engineers on system design, evaluation, and iteration
• Define and evolve evaluation frameworks across offline metrics, online experiments, and human feedback
• Establish clear quality bars and feedback loops for AI-powered experiences
• Drive execution through clear specifications, prioritization, and strong technical judgment
• Identify failure modes and ensure Artificial Intelligence (AI) systems behave predictably and recover gracefully
• Own product quality end-to-end, including correctness, reliability, usefulness, and user trust
Requirements:
• Artificial Intelligence (AI) and Machine Learning (ML) experience required
• Strong computer science fundamentals and system design skills
• Solid understanding of modern Machine Learning (ML) and how Artificial Intelligence (AI) systems behave in production
• Comfortable reading technical designs, understanding architectures, and engaging deeply with engineers
• Strong intuition for model limitations, hallucinations, evaluation, and failure modes
• Significant experience owning complex, technical products end-to-end
• Hands-on experience with AI-powered products, particularly LLM-based systems
• Experience with model evaluation, experimentation, prompting, or Artificial Intelligence (AI) system iteration
• Ability to turn ambiguous problems into clear requirements, measurable outcomes, and executable plans
• Strong judgment and ability to make decisions when the right answer is not obvious
• Highly technical, curious, and comfortable going deep
• Strong bias toward shipping, experimentation, and learning from real-world usage
• Able to challenge engineers and researchers constructively while maintaining strong collaboration
• Comfortable operating with significant ownership in a fast-moving, zero-to-one environment
• MUST BE WILLING TO TAKE A 60 MINUTE CODING ASSESSMENT
• Engineering background is a strong plus
• Experience shipping AI-heavy consumer products is nice to have
• Background as an engineer or highly technical product manager is nice to have
• Experience defining evaluation metrics for Machine Learning (ML) systems is nice to have
• Strong intuition for Artificial Intelligence (AI) UX patterns and failure handling is nice to have
• Prior experience in zero-to-one product environments is nice to have
Benefits:
• Medical insurance
• Dental insurance
• Vision insurance
• Savings Plan Options
• PTO
• Remote work / work from home
Apply To This Job