Note: The job is a remote job and is open to candidates in USA. Microsoft is seeking a detail-oriented and customer-focused Content QA Analyst to ensure the quality, reliability, and effectiveness of content experiences across Microsoft 365. The role develops test plans, executes manual and automated tests, evaluates AI-generated outputs, tracks defects, and provides recommendations to improve content and customer experience.
Responsibilities
- Develop and execute test plans for templates, prompt templates, AI-generated content, and other Microsoft 365 content experiences
- Perform manual and automated testing across Word, Excel, PowerPoint, Create, and Copilot experiences
- Validate content quality, functionality, usability, accessibility, and adherence to design fidelity
- Identify, document, reproduce, and track defects through resolution
- Evaluate AI-generated outputs against defined quality standards and acceptance criteria
- Assess content accuracy, relevance, usability, formatting, and customer experience
- Provide clear recommendations to improve model and content performance
- Create and maintain test scripts, test cases, validation frameworks, and execution documentation
- Develop repeatable testing approaches for new content experiences and features
- Partner with content and cross-team stakeholders to identify opportunities for automation
- Analyze testing results and communicate findings to stakeholders
- Identify patterns, recurring issues, and opportunities for quality improvements
- Track testing outcomes and ensure changes are reflected in documentation and content production workflows
Skills
- 2-4 years of experience in QA, content testing, usability testing, or quality assurance
- Bachelor's degree in Content Automation, Computer Science, Engineering, Information Systems, or related field
- Experience creating test plans, test cases, and test reporting documentation
- Experience scripting in Python, PowerShell, or similar languages
- Knowledge of Microsoft 365 products and productivity workflows
- Strong understanding of testing methodologies and quality processes
- Experience testing AI-generated content or generative AI experiences
- Understanding of design principles, content quality, and user experience
- Experience working with automation frameworks
- Familiarity with human evaluation methodologies and quality rubrics
- Excellent analytical and problem-solving skills
- Strong written and verbal communication skills
Benefits
- Medical, dental & vision
- Hospital plans
- 401(k) Retirement Plan – Pre-tax and Roth post-tax contributions available
- Life Insurance (Company paid Basic Life and AD&D as well as voluntary Life & AD&D for the employee and dependents)
- Company paid short and long-term disability
- Health & Dependent Care Spending Accounts (HSA & DCFSA)
- Employee Assistance Program
- Time Off/Leave (PTO, Allegis Group Paid Family Leave, Parental Leave)
Company Overview