Note: The job is a remote job and is open to candidates in USA. Handshake is an AI data business that works with frontier AI labs to create evaluations, publish benchmarks, and advance AI capabilities. The CBRNE Red Teamer evaluates AI models for dangerous gaps in their handling of chemical, biological, radiological, nuclear, and explosive threats. The role involves designing adversarial scenarios, assessing model outputs, documenting risks, and contributing to AI safety evaluation frameworks.
Responsibilities
- Design technically grounded adversarial prompts that test whether models provide meaningful uplift toward CBRNE threats
- Evaluate model outputs for technical accuracy, assessing whether responses contain genuinely dangerous information versus superficial or publicly available knowledge
- Probe dual-use knowledge boundaries, testing how models handle queries that blend legitimate scientific, medical, or industrial use cases with potential weapons applications
- Test multi-step and multi-turn attack chains that simulate how a motivated actor might extract dangerous information incrementally
- Score model responses against structured harm taxonomies and severity rubrics calibrated to real-world risk
- Document findings with clear technical reasoning, including what a response gets right, what it gets wrong, and why the failure matters
- Identify and articulate the difference between information that is freely available in open literature and information that constitutes genuine uplift beyond baseline
- Contribute to the development and refinement of CBRNE-specific evaluation frameworks and threat models
- Collaborate with other red teamers, AI researchers, and policy teams to translate findings into actionable model improvements
- Stay current on evolving model capabilities, jailbreak techniques, and relevant developments in your domain
Skills
- Graduate-level education or equivalent professional experience in a relevant CBRNE field (chemistry, biochemistry, microbiology, virology, nuclear physics, radiochemistry, materials science, munitions/ordnance, chemical engineering, or closely related disciplines)
- Ability to evaluate the technical accuracy and real-world consequence of model outputs in your domain
- Understanding of dual-use research concerns and the distinction between open-source knowledge and operationally significant uplift
- Strong hands-on experience using multiple LLMs (ChatGPT, Claude, Gemini, open-source models, etc.)
- Creative, adversarial problem-solving skills
- Clear and precise written communication, including the ability to explain technical risk to non-specialist audiences
- Strong ethical judgment and the ability to separate adversarial thinking from personal values
- Self-directed, collaborative, and comfortable in feedback-heavy environments
- Candidates must be able to engage with this material professionally and sustainably
- Active or prior security clearance (Secret, Top Secret, or SCI)
- Experience in threat assessment, WMD analysis, intelligence analysis, or arms control verification
- Background in biosafety/biosecurity, chemical safety, nuclear nonproliferation, or explosive ordnance disposal
- Familiarity with relevant regulatory frameworks (CWC, BWC, IAEA safeguards, ATF regulations, Export Administration Regulations)
- Experience in red teaming, penetration testing, or structured adversarial evaluation in any context
- Familiarity with Python or scripting languages, LLM APIs, or evaluation tooling
- Published research or professional presentations in a relevant CBRNE domain
- Prior work in trust and safety, content moderation, or AI evaluation
Benefits
- Remote work
Company Overview
Company H1B Sponsorship