Note: The job is a remote job and is open to candidates in USA. Terac is building a pool of vetted human experts for AI research and product teams. The company is seeking software engineers for a paid remote study evaluating the quality, realism, complexity, and technical accuracy of programming tasks and AI evaluation harnesses. Participants will identify flaws and provide feedback against real-world engineering standards.
Responsibilities
- Review and assess the quality of realistic programming tasks
- Evaluate coding environments and technical harnesses for AI testing
- Walk us through potential flaws in the provided code structures
- Provide feedback on the difficulty and realism of the challenges
Skills
- • Professional experience as a software engineer or developer
- • Familiarity with building or verifying programming tasks
- • Experience with code review and evaluation harnesses
- • Comfortable discussing technical architecture and testing methodologies
Benefits
- Remote session
Company Overview