We are sharing a specialised part-time consulting opportunity for experienced QA professionals with strong expertise in software testing, quality assurance, test-case design, bug analysis, and manual or automated testing to contribute to an advanced AI training and technical evaluation project.
Selected professionals will evaluate AI-generated technical outputs, design and assess comprehensive test scenarios, review defects and documentation, and provide detailed written feedback that supports improvements in software and AI system quality. The work combines traditional QA practices with structured evaluation and rubric-based assessment. No prior experience in AI is required.
Key Responsibilities
AI Output & Quality Evaluation
Evaluate and rate AI-generated technical outputs against defined quality criteria and evaluation rubrics
Identify responses that appear correct at first glance but contain inaccuracies, omissions, or incomplete reasoning
Assess outputs for technical correctness, completeness, clarity, and adherence to requirements
Apply consistent judgement across high-volume evaluation workflows
Document quality concerns and recurring failure patterns clearly
Test Case Design & Software Validation
Design and assess comprehensive test cases covering functional, regression, negative, boundary, and edge-case scenarios
Review software behaviour against expected requirements and acceptance criteria
Apply systematic testing approaches to identify defects and reliability issues
Evaluate both common and less obvious failure conditions
Maintain strong coverage and consistency across manual and automated testing workflows
Bug Analysis & Defect Documentation
Review bug reports and test documentation for reproducibility, completeness, and appropriate severity
Identify, isolate, and document software defects with precise reproduction steps
Ensure issue descriptions contain sufficient technical detail for efficient investigation
Use structured bug-tracking and test-management processes to record findings
Assess whether reported defects accurately reflect system behaviour and impact
Automation, Feedback & QA Collaboration
Work with automation frameworks and testing tools to support efficient quality evaluation
Provide detailed, actionable written feedback and annotations that developers can use without additional clarification
Collaborate with project teams to refine testing standards, rubrics, and evaluation guidelines
Help improve QA methodologies and consistency across project workflows
Contribute to continuous improvement of testing processes and technical quality standards
Ideal Profile
Proven professional experience as a QA Engineer, SDET, Test Engineer, QA Analyst, or in a closely related software quality role
Strong understanding of QA fundamentals including test-case design, regression testing, bug tracking, and manual and automated testing strategies
Hands-on experience with testing or automation tools such as Selenium, Playwright, Cypress, Appium, Postman, Jira, TestRail, Zephyr, BrowserStack, or comparable platforms
Strong analytical and problem-solving skills with meticulous attention to detail
Ability to identify subtle defects, ambiguous behaviour, incomplete requirements, and inconsistent system outputs
Strong written English communication skills at B2 level or above
Ability to provide precise, actionable technical feedback
Prior paid experience with human data annotation, RLHF, AI response evaluation, model evaluation, or rubric-based grading is advantageous but not required
Practical, demonstrable QA experience is prioritised over formal academic credentials
Reliable internet connection and readiness to begin promptly
No prior experience in AI research or model training is required
Engagement Details
Part-time independent contractor engagement
Fully remote
Compensation: $90–$175/hour
Work will involve AI-output evaluation, test-case design, manual and automated testing, bug analysis, defect documentation, and structured QA feedback
Strong professional software-testing experience is central to this engagement
Prior human-data or AI-evaluation experience is beneficial but not mandatory
No formal degree is required
Project scope, workload, timing, and evaluation standards may evolve depending on project requirements
Work must be completed without using confidential or proprietary information belonging to any employer, client, institution, or other third party
About the Platform
This opportunity is available through 24-MAG LLC. We connect experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams.
By submitting this application, you acknowledge that your information may be processed by 24-MAG LLC for recruitment and opportunity matching in accordance with our Privacy Policy: https://www.24-mag.com/privacy-policy