Note: The job is a remote job and is open to candidates in USA. 24-MAG LLC connects experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams. The Senior Software Engineer will develop reinforcement-learning environments for advanced AI evaluation, integrate Model Context Protocol tools, and create realistic software-engineering tasks involving debugging, feature development, refactoring, performance optimization, verification, and code review.
Responsibilities
- Design realistic software-engineering environments for AI evaluation
- Create tasks involving bug fixing, feature development, refactoring, and performance optimisation
- Build reproducible environments with clearly defined expected behaviour
- Develop technical scenarios that require structured reasoning over unfamiliar systems
- Ensure tasks accurately measure both software-engineering ability and tool use
- Design tasks requiring agents to interact with real MCP servers
- Create workflows that require discovery and retrieval of relevant information through MCP tools
- Evaluate whether agents use available tools appropriately and efficiently
- Build scenarios combining external context with software-engineering problem solving
- Define expected MCP interaction patterns where required for task completion
- Work across C++, Python, Java, Go, TypeScript, Rust, or comparable languages
- Diagnose complex defects and identify root causes
- Implement maintainable fixes rather than superficial patches
- Analyse unfamiliar codebases and dependencies
- Apply strong software-engineering practices throughout task development
- Design and implement realistic feature-development scenarios
- Refactor existing code to improve clarity, maintainability, and architecture
- Preserve intended functionality while improving implementation quality
- Evaluate alternative design and implementation approaches
- Ensure solutions remain scalable and maintainable
- Develop tasks involving software-performance bottlenecks
- Analyse algorithmic and implementation-level inefficiencies
- Improve latency, throughput, or resource utilisation where appropriate
- Validate that optimisations preserve correctness
- Document performance reasoning and technical trade-offs clearly
- Develop deterministic verification mechanisms for each environment
- Create golden reference solutions demonstrating expected task completion
- Define objective acceptance criteria for AI-generated outputs
- Ensure tests reliably distinguish correct from incomplete or incorrect solutions
- Validate both tool usage and software-engineering outcomes
- Review technical submissions for correctness, maintainability, and clarity
- Identify hidden defects, unsupported assumptions, or incomplete implementations
- Apply rigorous software-engineering standards to evaluation tasks
- Contribute to development of consistent technical-quality criteria
- Incorporate review feedback into evolving environment designs
Skills
- Strong professional software-engineering experience
- Proficiency in one or more of C++, Python, Java, Go, TypeScript, or Rust
- Deep understanding of algorithms and data structures
- Strong performance-tuning and optimisation skills
- Demonstrated experience debugging complex software systems
- Strong background in feature implementation and codebase refactoring
- Proven ability to develop maintainable and scalable solutions
- Experience working with large or distributed codebases is highly valuable
- Strong code-review experience and familiarity with software best practices
- Excellent written and verbal technical communication skills
- Comfortable collaborating across cross-functional and remote teams
- Part-time independent contractor engagement
- Fully remote
- Expected commitment: approximately 15 hours per week
- Familiarity with modern AI or machine-learning systems is advantageous but not required
- No prior AI-training experience is required
Benefits
- Fully remote
Company Overview