# Software Engineering AI Evaluator
## About the Role
Mercor connects elite creative and technical talent with leading AI research labs. This remote contract role focuses on evaluating and improving AI model capabilities across complex software engineering tasks.
You will collaborate with AI research teams to improve training data quality, develop benchmarks for AI solutions, and assess model reasoning and performance. The role offers an opportunity to directly contribute to the development of advanced AI systems while working independently and asynchronously.
## Key Responsibilities
Evaluate AI model capabilities across complex software engineering tasks.
Assess and improve the quality of AI training data.
Collaborate with AI research teams to support model development and evaluation.
Develop and contribute to benchmarks for AI-powered software engineering solutions.
Analyze model reasoning and identify opportunities for improvement.
Help shape AI model performance through high-quality technical evaluation and feedback.
Work independently and asynchronously while meeting project deadlines.
## Required Qualifications
Strong software engineering or technical background relevant to complex SWE tasks.
Ability to evaluate technical AI outputs and reasoning.
Strong analytical and problem-solving skills.
Ability to work independently in an asynchronous environment.
Strong attention to data quality and technical accuracy.
Ability to meet deadlines and deliver high-quality work.
## Work Arrangement
Employment Type: Contract
Work Arrangement: Remote
Hours: 20 hours per week
Work Style: Independent and asynchronous
## Compensation
$150–$210 per hour
## About Mercor
Mercor connects highly skilled creative and technical professionals with leading AI research organizations, enabling experts to contribute to the development and evaluation of advanced AI systems.