Note: The job is a remote job and is open to candidates in USA. League is a healthcare experience platform building a Benefits Navigation Agent that helps members understand and use their health benefits. Freelance clinicians will review and evaluate AI-generated benefits guidance, design and pressure-test scenarios, validate assessment rubrics, and provide subject-matter feedback to ensure responses are accurate, complete, and trustworthy.
Responsibilities
- Scenario design and pressure-testing — building and stress-testing the member scenarios the agent is evaluated against, so the rubric reflects the questions members actually ask and the failure modes that matter most
- AI agent evaluation — supporting the development and validation of rubrics, grading AI-generated benefits guidance against structured accuracy and coverage standards, flagging where the agent misstates coverage or oversteps into advice it shouldn't give, and providing written feedback on how responses can be improved
- Subject-matter input — bringing a benefits, claims, or regulatory perspective to how we define correctness, coverage, and the hard-fail boundaries the agent must never cross
Skills
- * Meaningful hands-on experience in one or more of the specializations above (e.g., benefits administration, plan design, payer or claims operations, benefits regulatory/compliance, member advocacy, or supplemental/point-solution expertise)
- * Based in Canada or the United States, with working knowledge of the relevant benefits and insurance landscape
- * Strong command of where accurate benefits guidance ends and regulated advice (legal, tax, clinical) begins
- * Reliable, detail-oriented, and dependable on deadline-driven async work
- * Ability to access and operate within Google Workspace
- * Prior experience with evaluation or validation of AI products, or structured content/quality review
- * Familiarity with self-funded vs. fully-insured plan mechanics, accumulators, coordination of benefits, or claims adjudication
- * Genuine curiosity and interest about the role of AI in benefits navigation
Benefits
- Fully remote and async-friendly
- Steady, ongoing work available for experts who are a strong fit
- Flexible hours that work around your day job or other commitments
- Structured onboarding and calibration so expectations are clear from the start
- Toronto-area Leaguers living within 65 km of the downtown headquarters receive 10 or 20 Flexible Remote Days each quarter for focus and deep-work time
Company Overview
Company H1B Sponsorship