Note: The job is a remote job and is open to candidates in USA. Crossing Hurdles is seeking a Senior Software Engineer to support AI data generation, collection, and evaluation workflows. The contractor will build and maintain Python and TypeScript pipelines and infrastructure, develop preference-data rubrics, review and label data, collaborate with researchers and engineers, and document technical findings.
Responsibilities
- Build and maintain pipelines and infrastructure supporting AI data generation, collection, and evaluation workflows
- Design, refine, and iterate on rubrics and evaluation criteria for preference data
- Review and label preference data while identifying subtle quality issues
- Collaborate with researchers and engineers to translate qualitative judgment into scalable evaluation processes
- Synthesize technical findings into clear, actionable write-ups, updates, and recommendations
Skills
- 5+ years of experience with Python, TypeScript, Golang, or JavaScript, particularly building and maintaining pipelines or infrastructure
- Hands-on experience with preference data collection and/or rubric creation
- Experience navigating or contributing to large codebases
- A LinkedIn profile is mandatory for review
- Candidates should not currently be working with Microsoft
- Experience with codebases such as VSCode
- Experience with AI/LLM evaluation or related engineering workflows
Benefits
- Remote work
- Part-time commitment of 20 hours per week
Company Overview