We are working with a leading AI Data Training Lab and are looking for experienced English-fluent AI users based in the US to evaluate and compare responses from leading AI models side by side. You will work directly with different models and tools, judge their responses on several quality dimensions, and explain your reasoning clearly. This is a fully remote, flexible project for people who already use LLMs regularly and have a sharp eye for what makes a response good.
• Duration: 3 months
• Start Date: From 13 October 2026
• Pay rate: $16.66 per hour
• Location: Remote but must be based in the United States
What you will do
• Interact directly with different AI models and tools
• Compare two or more model responses side by side (SxS)
• Judge responses on several quality dimensions, such as accuracy, helpfulness, instruction-following, clarity and overall quality
• Follow multi-step evaluation workflows and detailed rating guidelines
• Write clear, well-reasoned explanations for each evaluation decision
• Record consistent judgments across a wide range of prompts and topics
What we are looking for
• Fluent English, with strong reading comprehension and writing skills
• Extensive hands-on experience using LLMs and evaluating their outputs
• Strong critical-thinking skills and the ability to compare responses on several quality dimensions
• Close attention to detail and careful instruction-following
• Able to explain evaluation decisions clearly and convincingly
• Comfortable following multi-step workflows and switching between different models and tools
How onboarding works
• Sign Up: If you are selected after applying, we will send you a separate email link to sign up for the project. - PLEASE RESPOND HERE for first steps
• Statement of Work: if you are eligible, you will be sent a Statement of Work (SoW)
• Certification: complete a certification assessment and meet the required pass rate
• Production: once certified, you will move into active task pools