Note: The job is a remote job and is open to candidates in USA. CodeGeniusRecruit is recruiting software engineering professionals to support the evaluation and improvement of AI coding agents. The role involves designing software engineering challenges, building Python reference solutions, using and assessing AI coding assistants, reviewing peer-created tasks, and analyzing agent failures.
Responsibilities
- Design realistic, multi-step software engineering challenges to test AI coding agents
- Build reference solutions in Python with clear, verifiable answers
- Utilize AI coding assistants in daily workflows and assess their effectiveness
- Review and refine tasks created by peers for clarity and correctness
- Analyze AI agent failures to improve task design and model evaluation
Skills
- MSc or PhD in computer science or a related STEM field, or equivalent practical experience
- Have strong relevant experience in research, research-engineering, or software engineering roles
- Proficiency in Python scripting and debugging with clean-code practices
- Fluency with version control systems, IDEs, and standard software development workflows
- Experience with AI coding assistants, prompt engineering, or agent workflows is preferred
Benefits
- Remote work arrangement
- Commission: $60-$90 per hour
Company Overview