Note: The job is a remote job and is open to candidates in USA. Microsoft's Cloud & AI organization works to secure digital technology platforms, devices, clouds, and AI systems across heterogeneous environments. The AI Security Engineer will design and execute cyber-capability evaluations for frontier AI models and agentic systems, develop realistic cyber environments and evaluation tooling, analyze model behavior, and communicate security findings.
Responsibilities
- Design, implement, and operate offensive cyber-capability evaluations for frontier, preview, production, and open-weight AI models
- Build and maintain realistic evaluation tasks covering vulnerability analysis, exploit development, application and system exploitation, attack-path reasoning, post-exploitation activities, and multi-step offensive workflows
- Execute controlled experiments using AI models; define success criteria and baselines; collect reliable telemetry; and document results
- Analyze model trajectories and investigate unexpected behavior to determine whether it reflects genuine capability, task leakage, environmental flaws, scoring errors, or other evaluation artifacts
- Develop and maintain cyber ranges, vulnerable applications, exploit-development targets, and evaluation harnesses, and apply security controls for sandboxing, secrets management, telemetry, access, and containment when testing autonomous AI systems
- Partner with red-team operators, researchers, engineers, and Responsible AI stakeholders to translate evaluation findings into security insights, and communicate results through reports, documentation, and briefings
Skills
- Master's Degree in Statistics, Mathematics, Computer Science, Computer Security, or related field AND 1+ year(s) experience in software development lifecycle, large-scale computing, threat analysis or modeling, cybersecurity, vulnerability research, and/or anomaly detection
- Bachelor's Degree in Statistics, Mathematics, Computer Science, Computer Security, or related field AND 2+ years experience in software development lifecycle, large-scale computing, threat analysis or modeling, cybersecurity, vulnerability research, and/or anomaly detection
- Equivalent experience
- Ability to meet Microsoft, customer and/or government security screening requirements are required for this role
- This position will be required to pass the Microsoft background and Microsoft Cloud background check upon hire/transfer and every two years thereafter
- This role will require access to information that is controlled for export under export control regulations, potentially under the U.S. International Traffic in Arms Regulations or Export Administration Regulations, the EU Dual Use Regulation, and/or other export control regulations. As a condition of employment, the successful candidate will be required to provide proof of citizenship, U.S. permanent residency, or other protected status (e.g., under 8 U.S.C. § 1324b(a)(3)) for assessment of eligibility to access the export-controlled information
- This position requires verification of citizenship due to citizenship-based legal restrictions. Specifically, this position supports United States federal, state, and/or local government agency customers and is subject to certain citizenship-based restrictions where required or permitted by applicable law
- Programming ability, particularly in Python, with experience building automation, scripts, or security tooling
- Familiarity with large language models, generative AI systems, coding models, AI agents, or model APIs
- Ability to follow experimental methodology, record results accurately, and document technical findings clearly
- Exposure to security labs, cyber ranges, capture-the-flag environments, or vulnerable applications
- Experience contributing to cyber-capability evaluations or model-evaluation work for AI or agentic systems
- Coursework, internship, competition, or project experience in exploit development, vulnerability research, penetration testing, application security, or red teaming
- Experience working with coding agents, tool-using models, or autonomous agent frameworks
- Experience building capture-the-flag challenges, vulnerable applications, or exploit-development targets
- Familiarity with PyRIT, adversarial-testing tools, or comparable model-evaluation frameworks
- Familiarity with containers, sandboxed execution, or cloud-based test infrastructure
- Hands-on cybersecurity experience with systems, networks, applications, vulnerability analysis, penetration testing, red teaming, or exploitation in authorized environments
- Experience working with large language models, generative AI systems, coding models, AI agents, model APIs, or model-evaluation frameworks
- Experience designing controlled experiments, defining success criteria, analyzing results, and documenting technical findings
- Familiarity with common evaluation risks, including contamination, task saturation, unreliable scoring, weak baselines, environmental leakage, and limited reproducibility
- Doctorate in Statistics, Mathematics, Computer Science, Computer Security, or related field OR Master's Degree in Statistics, Mathematics, Computer Science, Computer Security, or related field AND 3+ years experience in software development lifecycle, large-scale computing, threat analysis or modeling, cybersecurity, vulnerability research, and/or anomaly detection
- Bachelor's Degree in Statistics, Mathematics, Computer Science, Computer Security, or related field AND 5+ years experience in software development lifecycle, large-scale computing, threat analysis or modeling, cybersecurity, vulnerability research, and/or anomaly detection
- Equivalent experience
Benefits
- Certain roles may be eligible for benefits and other compensation.
Company Overview