Remote Data Annotation Jobs In San Diego — Role Overview
Rex.zone is hiring a Remote Data Annotation Specialist to create and evaluate labeled datasets that train and validate modern AI/ML systems. You will work across NLP labeling, computer vision annotation, RLHF preference ranking, and prompt-response evaluation while following versioned guidelines and meeting clear quality targets.
What You Will Do
• Execute data labeling tasks across NLP and computer vision annotation projects (text span labeling, classification, bounding boxes, segmentation, attribute tagging).
• Perform RLHF and large language model evaluation by ranking outputs, scoring prompt-response pairs, and identifying instruction-following failures.
• Run QA evaluation workflows including self-checks, peer reviews, and gold-standard audits to improve training data quality and consistency.
• Apply content safety labeling taxonomies and escalate uncertain cases using defined escalation paths.
• Write clear rationales, document edge cases, and propose guideline updates when recurring ambiguities appear.
• Track throughput and quality metrics (accuracy, agreement rates, defect density) to support model performance improvement.
Required Qualifications
• Mid-Senior experience in data annotation, data labeling, QA evaluation, or LLM evaluation in production or vendor environments.
• Strong reading comprehension and attention to detail for rubric-based judgment and policy-based labeling.
• Comfort with annotation tools, task queues, and strict annotation guidelines compliance.
• Ability to explain decisions with concise written rationales.
• Familiarity with NLP concepts such as named entity recognition and taxonomy-driven labeling (preferred).
Work Arrangement
This is a Remote, FULL_TIME role aligned to US-based projects. Collaboration is primarily asynchronous with periodic calibration and QA review sessions.
Location reference: San Diego, CA, US.
Apply To This Job