Note: The job is a remote job and is open to candidates in USA. 24-MAG connects experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams. The Subject Matter Expert will analyze complex statistical visualizations, design rigorous quantitative tasks, and produce objectively verifiable solutions for advanced AI benchmark and evaluation projects. The role also involves statistical inference, quality assurance, technical documentation, and remote collaboration.
Responsibilities
- Interpret complex statistical and data-science visualisations
- Analyse Sankey diagrams, heatmaps, calibration curves, residual plots, correlation matrices, and distribution plots
- Extract quantitative information accurately from information-rich graphics
- Identify meaningful patterns, relationships, and anomalies
- Apply appropriate statistical terminology throughout analysis
- Design and solve tasks requiring multiple analytical steps
- Perform calculations based on chart-derived information
- Evaluate relationships across variables and data series
- Apply statistical reasoning where required
- Produce clearly supported and reproducible conclusions
- Create clear, objective, and unambiguous analytical questions
- Design tasks requiring reasoning beyond surface-level chart reading
- Develop prompts involving comparison, interpolation, inference, and trend analysis
- Ensure sufficient information is available to reach a definitive answer
- Avoid subjective or unnecessarily ambiguous task construction
- Apply appropriate statistical reasoning to visual data
- Interpret confidence intervals and uncertainty indicators
- Assess distributional patterns and relationships
- Evaluate calibration, residual behaviour, and correlations
- Clearly distinguish observed patterns from inferred conclusions
- Analyse relationships encoded visually across complex charts
- Compare trends and patterns across multiple series
- Interpret scales, legends, annotations, and graphical encodings
- Evaluate changes across time, groups, or categories
- Identify misleading or unsupported interpretations
- Produce precise final answers
- Document analytical steps clearly
- Show explicit calculations when required
- Explain statistical or modelling logic
- Ensure reasoning can be independently verified
- Use exact data-science and statistical terminology
- Reference units, axes, legends, scales, and annotations accurately
- Maintain consistency between visualisation, question, and solution
- Avoid unsupported claims
- Preserve technical accuracy across all deliverables
- Create high-quality tasks for advanced AI evaluation
- Develop benchmark questions reflecting real analytical work
- Ensure tasks are objectively scorable
- Identify whether model outputs demonstrate correct quantitative reasoning
- Contribute domain expertise to rigorous AI assessment workflows
- Review tasks for ambiguity or hidden assumptions
- Verify calculations and expected answers
- Check statistical language for accuracy
- Ensure tasks test meaningful analytical capability
- Maintain consistent quality standards across benchmark materials
- Collaborate asynchronously with project coordinators and reviewers
- Incorporate feedback into task revisions
- Communicate analytical reasoning clearly in written and verbal formats
- Support consistency across distributed contributor teams
- Maintain reliable communication throughout project workflows
Skills
- * Bachelor's degree or equivalent professional experience in Data Science, Statistics, Applied Mathematics, Computer Science, Operations Research, or a related quantitative field
- * Minimum 2 years of professional, academic, or research experience involving statistical models, visualisations, or analytical reporting
- * Strong ability to interpret complex charts and graphical outputs
- * Advanced understanding of axes, legends, units, scales, confidence intervals, and statistical annotations
- * Strong multi-step quantitative reasoning ability
- * Experience interpreting distributions, correlations, residuals, calibration, and related statistical concepts
- * Ability to write clear, objectively answerable analytical questions
- * Strong written and verbal English communication skills
- * Ability to explain analytical processes precisely
- * Strong attention to detail and analytical rigour
- * Experience with technical documentation, publication, assessment design, or structured task development is advantageous
- * Background creating data-annotation tasks, benchmark questions, technical reports, or similar analytical materials is valuable
- * Bilingual proficiency in English and another relevant language is advantageous but not required
Benefits
- Fully remote
Company Overview