Note: The job is a remote job and is open to candidates in USA. CyberCoders is a profitable, established enterprise software company that is modernizing its engineering organization. They are seeking a QA Engineering Manager to lead the evolution of their quality engineering strategy, focusing on AI-assisted testing and building evaluation infrastructure for AI-powered features.
Responsibilities
- Own the quality engineering strategy and execution across multiple product lines
- Lead the transition from traditional automation to AI-assisted and AI-agent-driven quality engineering using tools such as Claude Code, Cursor, GitHub Copilot, or similar
- Design and build evaluation infrastructure for LLM-powered product features, including golden datasets, prompt regression testing, tool-call validation, LLM-as-judge evaluation, and automated quality gates
- Establish quality standards for both human-written and AI-generated code through automated evaluation and review processes
- Hire, mentor, and grow a team of QA Engineers and SDETs while establishing engineering best practices
- Partner closely with engineering leadership on architecture, scalability, and quality strategy across AWS-based microservices
- Define and measure quality through business-focused metrics such as release confidence, escaped defect rates, regression effectiveness, and mean time to detection
- Stay hands-on by reviewing AI-generated test code, contributing to automation frameworks, improving CI/CD pipelines, and solving complex technical challenges
- Champion modern AI-driven testing practices and continuously evaluate new technologies that improve engineering velocity and product quality
Skills
- 7+ years of experience in software engineering, software quality engineering, or test automation
- 2+ years of engineering management experience, including hiring, mentoring, coaching, and performance management
- Demonstrated hands-on experience using AI coding agents such as Claude Code, Cursor, GitHub Copilot, or similar to accelerate software testing and quality engineering
- Strong experience building automated testing frameworks using Playwright (preferred) or another modern end-to-end testing framework
- Experience with unit testing, API automation, integration testing, and CI/CD pipelines
- Strong understanding of cloud-native applications, AWS, and microservices architecture
- Ability to contribute to architectural discussions while remaining hands-on technically
- A track record of measurable impact, including reducing regression time, increasing automation coverage, improving release confidence, or reducing escaped defects
- Experience building evaluation infrastructure for LLM-powered applications
- Experience with golden datasets, LLM-as-judge methodologies, prompt regression testing, tool-call validation, or AI evaluation frameworks
- Experience leading a team through the transition from traditional QA to AI-assisted or AI-agent-driven testing with measurable results
- Experience testing AI agents, MCP-based workflows, retrieval systems, or other LLM-powered applications
Benefits
- Bonus
- Remote flexibility
- Competitive compensation and bonus opportunities for exceptional, hands-on AI quality engineering leaders
Company Overview