Note: The job is a remote job and is open to candidates in USA. 24-MAG LLC connects experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams. The Data Scientist will support advanced AI evaluation work by designing realistic data science and analytics scenarios, developing reference analyses and evaluation rubrics, and assessing machine learning, experimentation, data infrastructure, business intelligence, MLOps, and data governance workflows.
Responsibilities
- Develop realistic scenarios involving predictive modelling and machine learning development
- Create tasks covering feature engineering, model selection, training, validation, and performance evaluation
- Assess modelling approaches for statistical validity, scalability, and business relevance
- Identify weak assumptions, methodological issues, and potential sources of bias or leakage
- Produce reference analyses and model documentation demonstrating strong technical judgement
- Develop tasks involving statistical inference, A/B testing, experimentation, and causal analysis
- Evaluate experimental designs, metrics, sampling approaches, and statistical conclusions
- Create scenarios involving ambiguous or noisy real-world datasets
- Assess whether analytical conclusions are adequately supported by available evidence
- Produce reference analyses explaining methodological choices and limitations
- Create scenarios involving enterprise data pipelines, data transformation, and analytics infrastructure
- Apply familiarity with platforms such as Snowflake, Databricks, and comparable data environments
- Develop tasks involving SQL, Python, R, and large-scale data-processing workflows
- Evaluate data architecture and pipeline decisions for reliability, scalability, and analytical usability
- Identify data-quality, lineage, governance, and integration issues
- Develop scenarios involving enterprise reporting, dashboards, and business intelligence
- Create tasks using tools such as Tableau, Power BI, or comparable analytics platforms
- Evaluate metrics, visualisations, and analytical outputs for clarity and decision usefulness
- Translate complex analytical findings into actionable business insights
- Produce reference executive-level analyses and recommendations
- Develop scenarios involving deployment, monitoring, validation, and lifecycle management of machine learning models
- Apply familiarity with platforms such as SageMaker, Vertex AI, MLflow, or comparable tools
- Evaluate model-monitoring strategies, retraining approaches, and production performance
- Assess operational risks including model drift, data changes, and reliability issues
- Apply MLOps best practices to realistic enterprise machine learning environments
- Create tasks involving enterprise data strategy, analytics governance, and cross-functional decision-making
- Evaluate approaches to data quality, privacy, access, governance, and analytical consistency
- Develop scenarios involving competing stakeholder and business requirements
- Assess strategic trade-offs across technical capability, risk, cost, and organisational priorities
- Produce reference recommendations reflecting senior-level data and analytics judgement
- Design expert-level evaluation tasks across data science and analytics workflows
- Draft reference outputs demonstrating authentic professional technical judgement
- Develop rubrics that distinguish experienced data science reasoning from generic textbook or tutorial-level responses
- Review AI-generated outputs for statistical accuracy, technical quality, and practical relevance
- Identify subtle analytical errors, unsupported conclusions, and unrealistic technical recommendations
- Provide clear written rationale supporting evaluation decisions
Skills
- 5+ years of professional experience in data science, analytics, machine learning engineering, or a closely related field
- Experience as a Data Scientist, Analytics Lead, Machine Learning Engineer, or comparable technical professional
- Direct ownership of production machine learning systems, analytics initiatives, data products, or large-scale analytical projects
- Strong proficiency in Python or R and SQL
- Experience with platforms such as Snowflake, Databricks, SageMaker, Vertex AI, MLflow, or comparable technologies
- Familiarity with Tableau, Power BI, or other enterprise business-intelligence tools
- Strong understanding of statistical modelling, experimentation, A/B testing, model validation, and causal inference
- Practical experience with data pipelines, MLOps, data governance, and production analytics
- Ability to communicate complex technical and analytical reasoning clearly and precisely in writing
- Experience developing technical documentation, training content, evaluation rubrics, or model documentation is advantageous
Benefits
- Fully remote
- Flexible scheduling based on project requirements
Company Overview