Note: The job is a remote job and is open to candidates in USA. Apt is seeking a Quality Engineer, AI & Test Automation to modernize quality practices through automation-first, engineering-integrated, and AI-enabled approaches. The role designs and maintains automated tests, validates AI-enabled capabilities, integrates quality checks into CI/CD workflows, manages defects, and partners cross-functionally to improve release readiness and delivery confidence.
Responsibilities
- Develop test strategies, test plans, acceptance criteria, and quality approaches for software and AI-enabled capabilities in partnership with Product, Engineering, Architecture, and business stakeholders
- Translate requirements, user stories, workflows, APIs, data dependencies, and operational expectations into clear test scenarios and quality validation plans
- Identify quality risks early in the delivery lifecycle and recommend appropriate validation approaches, including automated, manual, exploratory, integration, regression, performance, accessibility, or AI-specific evaluation methods
- Provide clear release-readiness input based on test evidence, defect trends, quality metrics, risk assessment, and stakeholder expectations
- Design, build, maintain, and improve automated test suites for API, UI, integration, end-to-end, regression, data, and workflow validation
- Integrate automated tests into CI/CD pipelines and delivery workflows so quality signals are available earlier and more consistently throughout the development lifecycle
- Create reusable test data, utilities, fixtures, scripts, and automation patterns that can be used across multiple teams and products
- Improve test reliability, maintainability, execution time, coverage, and signal-to-noise ratio by reducing brittle scripts and eliminating repetitive manual validation where practical
- Support testing and evaluation of AI-enabled experiences, including agent workflows, generated responses, retrieval behavior, tool/API calls, escalation paths, guardrail behavior, and human-in-the-loop patterns
- Help define evaluation datasets, expected behaviors, failure modes, scoring rubrics, regression scenarios, and quality thresholds for AI-enabled capabilities
- Use AI-enabled testing tools where appropriate to generate test cases, analyze logs, triage defects, identify coverage gaps, summarize results, and accelerate quality workflows
- Partner with AI Quality, Architecture, Engineering, and Product teams to validate that AI-enabled capabilities are useful, reliable, explainable enough for the use case, and aligned to defined quality expectations
- Identify, document, reproduce, triage, and communicate defects with sufficient technical detail to support efficient resolution by engineering teams
- Analyze defect patterns, escaped defects, quality trends, release issues, and production feedback to identify root causes and improvement opportunities
- Collaborate with engineering teams to improve testability, observability, logging, error handling, and diagnostic capabilities across applications and AI-enabled workflows
- Contribute to retrospectives, quality reviews, and process improvements that reduce rework, improve release confidence, and increase delivery velocity
- Work closely with Product Managers, Senior AI Product Managers, Software Engineers, AI Engineers, Data Product Owners, Architects, Designers, Clinical and Operational stakeholders, and vendors/partners to align on quality expectations and release readiness
- Ensure quality criteria are embedded in requirements, design reviews, backlog refinement, development, testing, implementation, and post-release monitoring
- Support production validation, monitoring handoffs, post-release assessment, incident follow-up, and quality reporting for assigned products and use cases
- Communicate quality status, risks, defects, test results, and recommendations clearly to technical and non-technical stakeholders
- Create and maintain test documentation, automation standards, quality playbooks, release-readiness checklists, defect triage practices, and reusable testing patterns
- Help teams adopt automation-first and AI-enabled quality practices while clarifying where manual validation remains necessary due to risk, ambiguity, or workflow complexity
- Share knowledge with other quality, product, engineering, and business stakeholders to raise Client’s overall quality engineering capability
- Stay current on modern testing approaches, automation frameworks, AI evaluation practices, test data management, observability, and quality engineering methods
Skills
- Bachelor's degree in Computer Science, Engineering, Information Systems, Data Science, or related technical field
- Equivalent combination of education and relevant technical experience may be considered
- 5–8+ years of experience in quality engineering, software testing, test automation, SDET, software engineering, AI quality, or related technology delivery roles
- Hands-on experience designing and implementing automated tests for APIs, web or mobile applications, workflows, data validation, integrations, or end-to-end product experiences
- Experience working with engineering teams in agile, DevSecOps, CI/CD, or modern product-delivery environments
- Experience with defect triage, root-cause analysis, test evidence documentation, release readiness, and quality metrics
- Strong understanding of software testing concepts, including test design, test coverage, functional testing, regression testing, integration testing, data validation, exploratory testing, and release validation
- Hands-on experience with test automation frameworks, scripting, test data, CI/CD integration, and defect management tools
- Ability to validate APIs, backend services, user interfaces, data flows, workflows, and enterprise integrations
- Programming, scripting, or data analysis skills, preferably Python, JavaScript/TypeScript, Java, SQL, or comparable languages used to build or maintain automated test assets
- Familiarity with AI-enabled testing and evaluation concepts, including prompt/response validation, model-output evaluation, RAG or retrieval testing, agent workflow testing, hallucination risk, guardrails, and human-in-the-loop workflows
- Ability to use logs, traces, telemetry, monitoring, dashboards, test artifacts, and production feedback to investigate quality issues
- Exposure to AI-enabled products, machine learning systems, large language models, conversational experiences, agentic workflows, automation, or decision-support technologies preferred
- Healthcare, life sciences, financial services, or other regulated-industry experience preferred but not required
- Experience as a Software Development Engineer in Test (SDET), Quality Engineer, Test Automation Engineer, Software Quality Engineer, AI Quality Engineer, or similar technically oriented quality role
- Experience building or maintaining automation frameworks using tools such as Playwright, Cypress, Selenium, Appium, PyTest, Postman/Newman, JUnit, TestNG, or comparable frameworks
- Experience testing AI agents, copilots, chatbots, LLM-powered applications, workflow automation, RAG systems, tool-calling workflows, or other AI-enabled capabilities
- Experience creating evaluation suites, golden datasets, scoring rubrics, regression tests, synthetic test data, or quality dashboards for AI-enabled systems
- Experience with cloud, APIs, microservices, event-driven systems, data platforms, analytics, observability tools, or modern application architectures
- Experience supporting quality engineering transformation, reducing manual testing dependence, or introducing AI-enabled testing practices into product teams
Company Overview