Note: The job is a remote job and is open to candidates in USA. OKAYA INFOCOM is recruiting for an Annotation Data Engineer role; the hiring client is not identified. The role focuses on annotation quality standards, metrics, audits, and dataset validation for large-scale machine learning and AI programs.
Responsibilities
- Define and maintain annotation quality standards, guidelines, defect categories, and acceptance criteria
- Develop metrics and reporting for accuracy, consistency, completeness, rework, throughput, and reviewer agreement
- Design sampling, audit, escalation, and root-cause analysis processes for large-scale annotation programs
- Evaluate generated captions, reasoning outputs, and foundation model training datasets against defined quality criteria
- Analyze quality trends and partner with annotation operations, engineering, and AI teams on corrective actions
- Validate dataset releases and maintain traceable quality records, issue logs, and improvement plans
- Support calibration sessions, reviewer alignment, and continuous improvement of annotation workflows
Skills
- Demonstrated experience in annotation quality, machine learning data operations, quality engineering, or data quality supporting physical AI, robotics, computer vision, or machine learning programs
- Demonstrated experience developing quality metrics and performance measurement systems
- Demonstrated experience working with large-scale annotation programs
- Demonstrated experience evaluating generated captions, reasoning outputs, or foundation model training datasets
- Experience defining annotation instructions, sampling plans, acceptance criteria, defect taxonomies, and audit processes
- Ability to analyze quality trends, identify root causes, and coordinate corrective actions
- Working knowledge of SQL, Python, reporting, or data analysis tools used for quality measurement
Benefits
- Remote work
Company Overview