Remote Data Annotation Jobs Amsterdam (Remote)
Rex.zone connects experienced annotators with real-world AI/ML training workflows, including data labeling, RLHF evaluation, prompt evaluation, and training data quality improvements for large language models. This is a full-time remote role supporting distributed teams and rubric-driven annotation operations.
About The Role
As a mid-senior Data Annotation professional, you will deliver high-precision labels and human judgments used to train, fine-tune, and evaluate AI systems. Work spans NLP classification, named entity recognition, prompt evaluation for LLMs, RLHF preference ranking, computer vision annotation, and content safety labeling.
Key Responsibilities
• Annotate and review datasets for NLP, CV, and content safety tasks while following detailed rubrics
• Perform RLHF workflows (preference ranking, pairwise comparisons, justification capture)
• Execute QA evaluation: sampling plans, discrepancy analysis, and rework loops to reduce label noise
• Document edge cases and propose guideline updates to improve inter-annotator agreement
• Support prompt evaluation and model evaluation sets measuring helpfulness, correctness, and safety
• Maintain secure handling of sensitive content and adhere to confidentiality requirements
Required Qualifications
• 3+ years in data annotation, data labeling, QA evaluation, or model evaluation for AI/ML
• Experience with annotation guidelines compliance and quality calibration methods
• Familiarity with RLHF, prompt evaluation, or LLM evaluation concepts
• Strong analytical writing for issue reports and edge-case documentation
• Comfort working remotely with distributed teams and structured throughput/quality metrics
Compensation
Hourly rate: $30–$50/hr (USD), depending on scope and evaluation responsibilities.