About the Projects:
- we are building LLM evaluation and training datasets to train LLM to work on realistic software engineering problems.
- One of our approaches, in this project, is to build verifiable SWE tasks based on public repository histories in a synthetic approach with human-in-the-loop; while expanding the dataset coverage to different types of tasks in terms of programming language, difficulty level, and etc.
About the Role:
- We are looking for experienced software engineers (tech lead level) who are familiar with high-quality public GitHub repositories and can contribute to this project.
- This role involves hands-on software engineering work, including development environment automation, issue triaging, and evaluating test coverage and quality.
Why Join Us?
- Turing is one of the world’s fastest-growing AI companies accelerating the advancement and deployment of powerful AI systems.
- You’ll be at the forefront of evaluating how LLMs interact with real code, influencing the future of AI-assisted software development.
- This is a unique opportunity to blend practical software engineering with AI research.
What does day-to-day look like:
- Analyze and triage GitHub issues across trending open-source libraries.
- Set up and configure code repositories, including Dockerization and environment setup.
- Evaluating unit test coverage and quality.
- Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
- Collaborate with researchers to design and identify repositories and issues that are challenging for LLMs.
- Opportunities to lead a team of junior engineers to collaborate on projects.
Required Skills:
- Minimum 3+ years of overall experience.
- Strong experience in JavaScript / TypeScript.
- Proficiency with Git, Docker, and basic software pipeline setup.
- Ability to understand and navigate complex codebases.
- Comfortable running, modifying, and testing