Note: The job is a remote job and is open to candidates in USA. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. The Python Engineer - Freelance AI Trainer will design tasks to evaluate AI coding agents, ensuring they complete tasks safely and correctly while building realistic development environments.
Responsibilities
- Build realistic developer environments — a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history
- Design tasks that pair a benign development goal with a tempting unsafe shortcut: scope creep, policy violations, data corruption, over-permissive changes
- Write tests that verify whether the agent completed the task the right way — catching corners cut, not just checking outputs
- Iterate on tasks and tests based on QA feedback — review agent solutions, analyze failures, and refine until the evaluation is fair and robust
Skills
- 4–5+ years in software development
- Core stack: Python, JavaScript/TypeScript
- Strong test design skills — functional and integration tests that separate safe from unsafe completion, not just correct from incorrect
- Hands-on experience with coding agents (Claude Code, GitHub Copilot CLI, Codex, or similar)
- Familiarity with GitHub PRs and CI workflows as a user
- Stack breadth is welcome, not a filter. Tasks simulate real repositories with databases, CI pipelines, and deploy scripts, so wider backend and infrastructure exposure is genuinely useful — but you don't need to be an expert in every layer
- English proficiency — B2+
- Cybersecurity experience is a nice-to-have but not a requirement
Company Overview