Note: The job is a remote job and is open to candidates in USA. Workera is a skills intelligence platform that uses AI to help organizations understand, manage, and develop their talent. The Senior Data Scientist will own the end-to-end scoring measurement system, including evaluation harnesses, calibration, quality monitoring, analytics investigations, and AI governance evidence. The role partners with Assessment Science, Engineering, Design, and Data Engineering to improve scoring quality and build scalable AI-assisted measurement workflows.
Responsibilities
- Own scoring quality end to end: evaluator design, rubric anchoring, calibration, and accountability for accuracy against expert benchmarks
- Build and run the continuous evaluation harness: gold sets, bias diagnostics, drift detection, and the pre-release gate that every scoring change must clear
- Define and publish assessment quality KPIs (human to AI agreement, reliability, classification accuracy, bias indicators, latency, cost per assessment) on dashboards anyone in the company can reference
- Lead the analytics investigations behind assessment decisions: performance studies, impact simulations, root cause analysis when scores look wrong
- Ship measurement improvements end to end: implement and coordinate with tech, own the spec, the validation, and the quality bar in both cases
- Be Workera's liaison between AI governance and InfoSec: stay current on frameworks like GDPR and the EU AI Act, and turn that into the evidence we show in audits, security reviews, and enterprise diligence when customers ask how we use AI in our assessments
- Empower partners across Product, Engineering, and GTM to get the data and analysis they need autonomously, by building AI tooling and documentation rather than answering each request yourself
- Immerse yourself in Workera's platform, customers, and the problems we're solving. You'll shadow key workflows and understand how AI is embedded in day-to-day operations across teams
- Deliver at least one analytics investigation that changes a product decision
- Sit in on an enterprise or auditor conversation about score quality to see what evidence customers actually ask for
- Own your first meaningful deliverable and demonstrate end-to-end execution
- Troubleshoot scoring end to end independently
- Ship one measurable scoring improvement, validated against expert labelled data, through the pre-release gate
- Operate with full autonomy in your domain; your team relies on your judgment
- Have built or deployed at least one AI-assisted workflow that the team adopts
- Set the scoring roadmap: identify the gaps, prioritise what gets built, and drive those initiatives to completion with Engineering
- Build systems that improve themselves: evaluation loops that flag their own drift, calibration that updates on new expert labels, tooling that gets better as it is used
- Use AI daily. AI assistants, copilots, and automation tools are part of your stack -- not optional extras. We expect you to actively experiment with new tools and push the boundary of what's possible in your function
- Build your own leverage. Our marketers write code. Our PMs build automations. Our ops team deploys agents. If a workflow can be automated, you're expected to automate it
- Think in systems, not tasks. We value people who build repeatable, scalable solutions over people who grind through one-off work. Your goal is to make your function run smarter, not just harder
Skills
- You've driven production data science work independently
- 4+ years (or 3+ with demonstrated end-to-end ownership) of a production data project ideally in an AI startup or fast-moving product environment where you scoped investigations yourself rather than picking up defined tickets
- Python and SQL are assumed; you shouldn't need an engineer to run an analysis
- Real statistical and analytical depth. You can turn, "is our scoring reliable?" into a defined study with a defensible method and a clear answer fast, and without being handed the design
- Quantitative work with educational, learning, or assessment data or a clear appetite to go deep on it fast
- You've worked with LLM-based systems in production. Prompt design, evaluating model output against human judgment, and a real sense of where these systems fail
- You explain quantitative results to non-technical audiences, including executives and customers, without losing the nuance
- A plus, not a requirement: background in psychometrics or educational measurement
Company Overview
Company H1B Sponsorship