Remote Data Labeling Specialist (Phoenix)
Remote data labeling jobs in Phoenix are mid-senior, full-time roles focused on creating and validating training data for AI systems. You will label text, images, video, and audio; follow annotation guidelines compliance; run QA evaluation; and support RLHF and prompt evaluation to improve large language model evaluation and model performance improvement.
About The Role
You will produce high-quality labeled datasets that power AI/ML model training and evaluation. Day-to-day work includes labeling and reviewing tasks across NLP and computer vision, applying consistent taxonomy and edge-case handling, and documenting decisions to maintain training data quality.
What You Will Do
Execute data labeling for text, image, video, and audio; apply annotation guidelines and flag ambiguous cases
Perform named entity recognition, sentiment/intent tagging, and instruction-following judgments for LLM training pipelines
Support RLHF by ranking model responses, rubric-based scoring, and prompt evaluation
Conduct QA evaluation (spot checks, inter-annotator agreement checks, error categorization, rework coordination, final dataset sign-off)
Audit outputs for content safety labeling and policy-driven requirements
Required Qualifications
Mid-senior experience in data labeling/data annotation programs with measurable quality outcomes
Strong guideline interpretation, consistency, and clear adjudication notes for edge cases
Familiarity with QA evaluation concepts (inter-annotator agreement, error taxonomies)
Working knowledge of NLP and computer vision annotation (e.g., NER, bounding boxes/polygons)
Work Model
This is a
Remote
,
FULL_TIME
role aligned to Phoenix candidates and time zones as needed. You will collaborate asynchronously with data operations, QA reviewers, and engineering stakeholders using web-based annotation tools, RLHF rubrics, and QA dashboards.
Pay
Competitive hourly pay:
$30–$50 per hour
(USD).