Note: The job is a remote job and is open to candidates in USA. 24-MAG LLC offers an independent-contractor consulting opportunity for experienced software engineers contributing to AI training and software-engineering evaluation projects. The role involves creating coding tasks, building reference implementations and verifiers, developing and evaluating backend services, and reviewing technical solutions.
Responsibilities
- Create realistic software-engineering tasks involving feature implementation and bug fixes
- Design challenges that reflect professional backend-development scenarios
- Define clear technical requirements, constraints, and expected behaviour
- Ensure tasks are sufficiently complex to evaluate engineering reasoning
- Apply practical software-development experience throughout task creation
- Work across Java, Node.js, Go, Rust, TypeScript, Python, C++, or comparable technologies
- Develop and evaluate scalable backend services and APIs
- Implement complex features within established codebases
- Apply sound algorithms, data structures, and computer-science fundamentals
- Maintain strong standards for correctness, maintainability, and performance
- Identify and resolve software defects and system bottlenecks
- Diagnose unexpected behaviour across complex applications
- Refactor existing code to improve structure and maintainability
- Optimise implementation performance where required
- Evaluate trade-offs between reliability, complexity, scalability, and efficiency
- Develop deterministic verifiers for coding tasks
- Ensure verifiers accept reasonable valid solutions and reject incorrect implementations
- Design automated tests covering expected behaviour and edge cases
- Diagnose flaky or unreliable test behaviour
- Apply strong testing strategy and code-review judgement across assignments
- Review software-engineering outputs for technical accuracy and completeness
- Assess implementation quality against task specifications
- Identify weak reasoning, incomplete fixes, or inappropriate engineering approaches
- Provide precise technical feedback on submitted solutions
- Maintain consistent quality standards across repeated evaluation workflows
Skills
- Strong professional backend software-engineering experience
- Expertise in one or more of Java, Node.js, Go, Rust, TypeScript, Python, C++, or similar languages
- Track record of building scalable services and APIs
- Strong debugging and software-troubleshooting capabilities
- Experience identifying and resolving performance bottlenecks
- Proven ability to implement complex features in established applications
- Experience refactoring large or evolving codebases
- Strong understanding of algorithms and data structures
- Solid automated-testing and code-review instincts
- Ability to diagnose flaky tests and improve testing standards
- Strong analytical and problem-solving skills
- Clear technical written communication
- Ability to work independently in a flexible remote environment
- No prior AI-training or model-evaluation experience is required
- Experience designing test strategies or maintaining large test suites is particularly valuable
Benefits
- Fully remote
- Flexible schedule across weekdays or weekends
Company Overview