About The Company
Based in San Francisco, California, Turing is the world's leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing supports customers through two primary avenues: first, by accelerating frontier research with high-quality data, sophisticated training pipelines, and top-tier AI researchers specializing in software engineering, logical reasoning, STEM, multilinguality, multimodality, and agents; and second, by applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligence. Our solutions are designed to perform reliably, deliver measurable impact, and generate lasting results on the profit and loss statements of our clients.
Turing's innovative approach and commitment to excellence position us at the forefront of AI research and enterprise deployment, making us a leader in the rapidly evolving AI landscape.
About The Role
We are seeking a highly skilled Software Engineering Evaluator to join our team. In this role, you will play a critical part in creating and curating cutting-edge datasets for training, benchmarking, and advancing large language models. Your primary focus will be on evaluating and refining AI-generated code, ensuring that it meets standards of efficiency, scalability, and reliability.
You will work closely with researchers to develop high-quality code examples, provide precise solutions, and make necessary corrections. The role involves working across multiple programming languages, including Python, JavaScript (with ReactJS), C/C++, Java, Rust, and Go, with an emphasis on backend services, data pipelines, and machine learning infrastructure. Your expertise will directly contribute to enhancing enterprise-level AI-driven coding solutions, supporting the development of robust AI models capable of real-world applications.
Qualifications The ideal candidate will possess several years of experience in software engineering, with a minimum of three years in building and deploying production systems at high-scale organizations such as Google, Microsoft, Apple, Amazon, or Meta. A strong educational background from top computer science programs like Stanford, MIT, Carnegie Mellon, UC Berkeley, Georgia Tech, or comparable institutions is highly valued, though exceptional practical experience and skills are prioritized. Proficiency in Python, including frameworks, tooling, and best practices for production-grade software, is essential.
Candidates should have demonstrated experience in building full-stack applications, deploying scalable software solutions, and possessing a deep understanding of software architecture, design, debugging, and code review. Excellent communication skills, both oral and written, are required to articulate evaluation rationales clearly and effectively.
Responsibilities
Curate and develop high-quality code examples across multiple programming languages, primarily Python, JavaScript (including ReactJS), C/C++, Java, Rust, and Go, for training and benchmarking large language models.
Evaluate AI-generated code for efficiency, scalability, and reliability, providing detailed feedback and corrections as needed.
Collaborate with research teams and cross-functional stakeholders to improve AI-driven coding solutions and ensure they meet industry performance benchmarks.
Design and build automated verification tools and agents in Python that can assess code quality, identify error patterns, and verify solutions to software engineering tasks.
Hypothesize and evaluate various stages of the software engineering cycle—including prototyping, architecture design, API development, production deployment, and operational maintenance—to assess model capabilities.
Develop verification mechanisms that can automatically validate solutions, ensuring consistent quality and performance standards.
Participate in ongoing research and development activities to refine dataset quality and improve model evaluation processes.
Benefits
At Turing, we offer a flexible and dynamic work environment that values innovation and professional growth. Our team members enjoy the opportunity to work on cutting-edge AI research and enterprise solutions, collaborating with some of the brightest minds in the industry. We provide a supportive environment with access to advanced tools and resources necessary for success.
Although this role is on a contractual basis, it offers the flexibility to work remotely within the United States, with a commitment of a minimum of 10 hours per week and up to 40 hours.
Compensation is competitive, aligned with industry standards for contract roles, and designed to recognize your expertise and contribution to our pioneering projects. This position is ideal for professionals seeking to expand their portfolio in AI research and software engineering while contributing to impactful technological advancements.
Equal Opportunity
Turing is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees and applicants. We do not discriminate based on race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. We believe that diverse teams foster innovation and creativity, and we are dedicated to providing equal employment opportunities to all qualified individuals.