Note: The job is a remote job and is open to candidates in USA. Crossing Hurdles is seeking a Japanese LLM AI Quality Analyst to support the evaluation and personalization of AI-generated responses. The role involves creating and assessing multi-turn prompts, reviewing response quality, comparing model outputs, documenting evaluation rationales, and following data-handling guidelines during a short-term remote contract.
Responsibilities
- Create and evaluate multi-turn prompts based on personal context and experiences
- Assess AI responses for personalization, accuracy, relevance, and helpfulness
- Review responses for unsupported assumptions, incorrect personalization, and other quality issues
- Compare model responses and provide structured evaluations
- Write clear rationales explaining evaluation decisions and observations
- Review how relevant information is incorporated into responses
- Follow project guidelines and maintain appropriate data-handling practices
Skills
- Strong Japanese proficiency, including reading and writing
- Excellent analytical thinking and attention to detail
- Ability to evaluate nuanced AI-generated responses
- Strong written communication skills and ability to provide clear feedback
- Ability to create and evaluate multi-turn prompts
- Experience in AI evaluation, data annotation, content moderation, or related work is preferred
Benefits
- Remote work
- Flexible 20 - 40 hrs/week with a minimum 4-hour overlap with PST
- Short-Term Contract (4 months)
Company Overview