Note: The job is a remote job and is open to candidates in USA. Crossing Hurdles is seeking a Senior Software Engineer to support AI data generation, collection, and evaluation workflows. The contractor will build infrastructure and pipelines, develop evaluation rubrics, review preference data, collaborate with researchers and engineers, and communicate technical findings.
Responsibilities
- Build and maintain pipelines and infrastructure supporting AI data generation, collection, and evaluation workflows
- Design, refine, and iterate on rubrics and evaluation criteria for preference data
- Review and label preference data while identifying subtle quality issues
- Collaborate with researchers and engineers to translate qualitative judgment into scalable evaluation processes
- Synthesize technical findings into clear, actionable write-ups, updates, and recommendations
Skills
- 5+ years of experience with Python, TypeScript, Golang, or JavaScript, particularly building and maintaining pipelines or infrastructure
- Hands-on experience with preference data collection and/or rubric creation
- Experience navigating or contributing to large codebases
- A LinkedIn profile is mandatory for review
- Candidates should not currently be working with Microsoft
- Experience with codebases such as VSCode
- Experience with AI/LLM evaluation or related engineering workflows
Benefits
- Remote work
Company Overview