Job Description:
• Design realistic, expert-level cybersecurity prompts spanning areas such as application security, network security, cloud infrastructure, vulnerability management, and secure software development.
• Evaluate AI-generated responses for technical accuracy, completeness, and practical usefulness.
• Assess how models handle sensitive and dual-use cybersecurity scenarios, ensuring responses are responsible and aligned with project guidelines.
• Apply structured evaluation rubrics to annotate prompts and model outputs with consistency and precision.
• Provide clear, actionable feedback that helps improve the performance, safety, and reliability of frontier AI systems.
Requirements:
• Bachelor's or Master's degree in Computer Science, Cybersecurity, or a closely related technical field.
• 5+ years of professional software engineering experience at a high-growth startup or established technology company.
• Deep knowledge of cybersecurity principles, modern software systems, and secure development practices.
• Excellent technical reasoning and strong written communication skills in English.
• Sound judgment when working with sensitive or dual-use security topics.
• The ability to apply detailed guidelines while maintaining high-quality, consistent evaluations.
• Nice to have: Experience in offensive security, penetration testing, vulnerability research, security engineering, or a related discipline.
• Previous experience evaluating, reviewing, benchmarking, or red-teaming technical or AI-generated content.
• Familiarity with AI evaluation, model assessment, or human-in-the-loop data annotation workflows.
Benefits:
• Contract, project-based position.