Note: The job is a remote job and is open to candidates in USA. RLink Solutions is hiring LLM–AI Quality Analysts for a project evaluating the quality of personalized AI interactions. The role involves designing conversational prompts, assessing AI responses for relevance, grounding, helpfulness, and naturalness, comparing responses, and writing detailed evaluation rationales.
Responsibilities
- Design and execute multi-turn conversational prompts , typically 1–5 turns
- Evaluate how well AI responses use relevant personal context
- Identify incorrect personalization, unsupported claims, and flawed inferences
- Compare two AI responses side-by-side (SxS) and determine which response is better
- Evaluate responses for Grounding, Integration, Helpfulness, and naturalness
- Write clear and detailed rationales explaining your evaluation decisions
- Reference specific turns and examples when providing feedback
- Review model/debug information to verify how information sources were used
- Maintain data hygiene by following project procedures for evaluation conversations
Skills
- Excellent written and spoken English
- Strong analytical and critical-thinking skills
- Ability to identify subtle differences between AI-generated responses
- Strong attention to detail
- Excellent written communication and ability to provide structured feedback
- Comfortable working independently in a remote environment
- Desktop/laptop with a reliable internet connection
- Candidates must be willing to use their primary personal Google account for the evaluation and enable the required personal data sources
- 1+ year of relevant experience preferred
- Experience in AI evaluation, data annotation, content moderation, quality assurance, research, or a related field is strongly preferred
- A Bachelor's degree or equivalent experience in a relevant field is preferred, including:
- Policy, Law, Ethics, Linguistics, Journalism, Computer Science, or another analytical/research-oriented field
Benefits
- Remote work arrangement
- Short-term contractor opportunity for 4 months
- Full-time/Part-time schedule
Company Overview