Note: The job is a remote job and is open to candidates in USA. Handshake is an AI data business that supports frontier AI labs by creating evaluations, benchmarks, and training data. The AI Model Policy Trainer will evaluate model interactions involving violence and related content, distinguish legitimate fictional or educational uses from harmful intent, write policy-based rationales, develop adversarial prompts, and identify policy gaps through calibration and review.
Responsibilities
- Evaluate user requests and AI model responses involving violence, weapons, threats, and dark fiction within the full conversation context
- Distinguish fictional, educational, historical, and defensive violence from requests that seek real-world uplift or express real intent to harm
- Assess whether a model's response gives meaningful real-world capability, regardless of how the request was framed
- Distinguish expressions of anger, frustration, or dark humor from credible threats or crisis indicators
- Select the most defensible classification when a case is genuinely ambiguous, and write concise rationales that cite policy language and conversation details
- Write and refine adversarial or borderline prompts that probe where a model draws the line
- Identify policy gaps, contradictions, and emerging edge cases, and raise them with project leads and policy teams
- Participate actively in calibration discussions; challenge interpretations respectfully and update your judgment when stronger reasoning emerges
- Apply customer policy consistently without substituting personal beliefs for the policy standard
- Maintain accuracy and attention to detail across repeated, feedback-heavy evaluations
Skills
- You need one deep and the judgment to learn the rest
- You can hold a strong opinion without becoming attached to being right
- You explain judgment calls clearly enough that another person can audit your reasoning
- You can separate your personal views from the standard a customer has asked you to apply
- You remain careful and consistent during repetitive work with difficult material
- You communicate clearly and precisely in writing
- A degree, a clearance, and a technical background are not required
- Candidates must be able to engage with this material carefully, responsibly, and sustainably while maintaining sound judgment and consistent work quality
- Location: Remote, US
- Schedule: 8AM - 5PM PT
- Weekly commitment: M-F
- You have spent serious time in violent fiction as a writer, game master, game designer, screenwriter, or editor, and you know what good dark fiction looks like and what a story-shaped extraction attempt looks like
- You have real-world exposure to violence and its consequences through military, law enforcement, security, EMS, emergency medicine, or similar work, and you can tell movie logic from what actually works
- You have worked with people in distress through crisis lines, counseling, threat assessment, domestic violence advocacy, school safety, or trust and safety, and you know the difference between "I could kill him" and a plan
- You use AI tools heavily and have opinions about where they refuse too much, help too much, or miss the point
- You notice when one word, contextual detail, or change in intent materially affects the answer
- Strong candidates may come from fiction writing, game design or game mastering, film and TV, military or law enforcement, emergency medicine, crisis counseling, threat assessment, trust and safety, content moderation, journalism, or law
- Published or produced work involving violence: novels, short fiction, screenplays, comics, tabletop or video game content, mods, or fan fiction with an audience
- Military, law enforcement, corrections, private security, or armed professional experience
- Training or professional experience in crisis intervention, threat assessment, forensic or clinical psychology, or violence prevention
- Experience with firearms, martial arts, or other weapons disciplines as an instructor, competitor, or professional
- Prior work in AI evaluation, red teaming, data annotation, RLHF, trust and safety, or content moderation
- Experience evaluating outputs from ChatGPT, Claude, Gemini, or other language models in a professional capacity
- Familiarity with calibration sessions, inter-rater agreement, or adjudication workflows
- Prior AI evaluation experience is helpful, but it is not required
Benefits
- Remote work arrangement
- Exposure limits and content rotation
- Access to mental health support
Company Overview
Company H1B Sponsorship