Note: The job is a remote job and is open to candidates in USA. AWeber is a remote-first company that helps small businesses build meaningful customer connections and grow through marketing communication software. The QA Engineer II will own quality end to end by building and reviewing agent-generated tests, developing automated coverage, performing exploratory testing, monitoring live systems, diagnosing failures, and determining release readiness.
Responsibilities
- You build the agents that do that work
- You own quality end to end: the test strategy, the automated coverage, the exploratory testing that finds what automation never will, real-time monitoring of live systems, and the call on whether something is ready to ship
- Your mission is to make our quality signals trustworthy, so that a green build actually means something and the team can move fast on top of it
- When you find a gap in the process, we expect you to fix it and tell people, not just file it
- Writing the test plan for a feature before a line of it exists
- Reviewing an agent-generated test suite and rejecting it because the assertions do not prove the behavior they claim to
- Exploring a new release by hand and finding the edge case nobody specified
- Tightening an agent's harness so it stops producing flaky waits
- Diagnosing why a pipeline job fails only in staging
- Writing up the cause of a failure so the next person does not have to rediscover it
Skills
- 4+ years in QA or test automation in an Agile environment, including hands-on experience building test automation frameworks and not only writing scripts inside someone else's
- API testing with Python, using pytest, unittest, or a comparable framework
- UI testing with Cypress and TypeScript, including the discipline of stable selectors and reliable page objects
- Demonstrated use of AI development tools and coding assistants to produce, review, or maintain tests, and the judgment to evaluate what they generate rather than accept it
- Solid understanding of CI/CD pipelines, ideally GitHub Actions, with the ability to diagnose a failing job rather than just report it
- Exploratory testing skill: finding meaningful defects in a running product without a script to follow
- Experience with Docker and containerized environments, Git and modern version control workflows, and API tooling such as Postman or curl
- Working experience with AWS services in a testing or deployment context
- Ability to apply proven QA techniques including requirement traceability, boundary value analysis, and application stability and usability testing
- Strong written communication. You will document test plans, failures, and sign-offs for an audience that includes both engineers and agents
- Proven ability to work self-directed in a deadline-driven environment, and the willingness to help more junior QA engineers on testing platforms and procedures
- Experience building agents and not only using them: writing and iterating on prompts, wiring up tool use, and getting a multi-step agent loop to run reliably against a real system
- Experience evaluating agent or LLM output systematically, whether through eval harnesses, scoring rubrics, or regression detection
- Experience testing in a high-volume agentic or AI-assisted codebase, including the verification and review challenges that come with it
- Experience with real-time monitoring and observability tooling, and using production signals to find quality problems before customers report them
- Experience with Kubernetes, or testing services deployed across multiple environments with differing states
- Familiarity with performance, load, or security testing
- Experience in a remote-first, asynchronous engineering culture
- Background in one of our domains: email or marketing SaaS, web platforms, or backend services at scale
- Working knowledge of the Atlassian stack (Jira, Confluence)
Benefits
- 100% Remote
- Strong culture that supports flexibility, entrepreneurialism, and collaboration.
- 100% Company Paid PPO medical, dental, vision insurance (spousal and domestic partner benefits available).
- 4-7 weeks of paid time off and holidays (based on tenure).
- 4 week paid sabbatical (based on tenure).
- 401K retirement plan with 4% company match.
- Company Profit Share.
- Home office equipment and internet stipend.
- Tuition reimbursement, conferences, and learning opportunities.
- Gym Memberships Reimbursement.
- Company Paid Short Term Disability Insurance.
- Company Paid Life Insurance.
- Additional Supplemental Benefits (Long Term Disability, Critical Illness, and Additional Life Insurance).
Company Overview
Company H1B Sponsorship