GovCIO is currently hiring a Software Development Engineer in Test (SDET), Test Engineer, to join cross-functional product teams building and modernizing federal digital services on AWS GovCloud.
This position is in the Continental United States (CONUS) and remote.
Job Description
A GovCIO SDET works in a cross-functional product team where everyone owns quality. The SDET plans testing for each story, builds and maintains automated tests, and reports what is covered and what is not. The role is hands-on, automation-first, and deadline-driven. Story testing starts and finishes inside the sprint; the SDET also creates the longer tests that span sprints, such as functional, regression, end-to-end, integration, performance, and load. A GovCIO SDET is not a script executor, not a click-through tester, and not the last gate before release. Quality starts at refinement: the SDET makes acceptance criteria testable, works from user stories without waiting for a BRD or FRD, and audits the tests developers write. The SDET knows what to regression test and where to look when something breaks, and can evaluate non-deterministic output using measurable thresholds that separate model variance from real defects.
Responsibilities
General
Designs, develops, and implements testing methods and equipment to ensure that the product will perform to specifications. Plans and arranges the labor, schedules, and equipment required for testing and evaluating the product. Designs test cases for test plans and creates test scenarios in which tests are carried out. Compiles data, defines required changes and reports defects and problems that occurred during the test process. Implements and participates in the walkthrough of the product. Automated testing may be used for leveling based on hiring manager discretion.
- Consults with the development and/or requirements team to coordinate and test applications.
- Performs analysis, diagnostics and preliminary evaluations of products. Develop testing procedures.
- Develops test plans, scripts, scenarios, functional tests, regression tests, and deployment tests.
- Implements the testing processes and ensures all test activities follow those processes. Produces reports on each test activity and tracks test metrics.
- Monitors for continual test process improvement through failure analysis, throughput analysis, yield analysis, and report to management periodically.
Specific
- Work as a full member of a Scrum team. Take part in sprint planning, backlog refinement, and design walkthroughs. Make user stories and acceptance criteria testable. Write your own stories and tasks.
- Design and run positive and negative tests: functional, integration, regression, API, and end-to-end. Find or create the test data. Pace the work so testing finishes inside the sprint.
- Build and maintain the automated test suites. Use Cypress and/or Playwright for the front end, Postman or scripted tests for REST APIs, and Locust for load and performance. Put the suites in the CI/CD pipeline with the DevOps engineers. Build and manage test environments on AWS GovCloud.
- Use Amazon S3 to check test data and artifacts. Use Amazon CloudWatch and Logs Insights queries for logs, metrics, and troubleshooting during test runs.
- Own regression testing for the product. Know how the system fits together. Decide what must be re-tested and in what order. Keep the regression suite current as the product changes.
- Audit the unit, contract, integration, and end-to-end tests the developers write. Work with them to improve coverage and quality.
- Check data in PostgreSQL and other data stores with SQL. Check JSON, XML, and file content with scripts.
- Test machine learning and deep learning features, especially the large language models and vision models behind OCR and intelligent document processing (IDP). Build and maintain ground-truth sets. Measure accuracy at the character, field, and document level and as a straight-through rate. Set threshold-based acceptance criteria and be able to say where each number came from. Check whether confidence scores predict correctness.
- Evaluate generative AI features. Run eval harnesses against versioned prompts. Test schema conformance and answer correctness as separate questions. Catch values the model made up that are not in the source document.
- Automate eval runs in CI so regressions show up before UAT.
- Keep the Requirements Traceability Matrix, defect records, test results, and coverage and defect summary reports current in Jira, Xray, and Confluence.
- Perform root cause analysis and defect isolation with the developers.
- Post short status in Jira every day, in plain English, that shows how the work contributes to the sprint goal and where it fits in the big picture. Every engineer on the team is expected to address anything that puts product quality, the sprint goal, or the release date at risk.
- Support UAT and Section 508 staging reviews. Support Authority to Operate (ATO), security, and compliance work. That includes PHI/PII safeguards in test data and evaluations.
- Earn and maintain AWS certifications as part of ongoing professional development.
Qualifications
Required Skills and Experience
- Bachelor's with 2-5 years (or commensurate experience)
- The candidate must complete the paperwork promptly and pass the background check.
- Must be a U.S. citizen or permanent resident.
- Must reside and work within the Continental United States (CONUS).
- Test automation (minimum): Cypress or Playwright plus at least one other framework. Selenium alone is not enough. A candidate who has not used Cypress must explain the architectural difference between Cypress and other frameworks such as Selenium or Playwright and must have the front-end skills to explain the implementation difference.
- Automated REST API testing with Postman or scripted tests.
- SQL for data validation. Python for test scripting and for working with data at scale.
- Able to test systems built on machine learning and deep learning, especially large language models and vision models. Understands non-determinism, ground truth, and threshold-based acceptance.
- ML: training versus inference, overfitting, class imbalance, and gaps between lab results and production results.
- Experience testing in AWS environments, including S3 and CloudWatch.
- Agile testing in Scrum: story refinement, testability of acceptance criteria, definition of done, and pacing tests to close the sprint.
- Jira, Xray, and Confluence for test management and reporting.
- Full range of test techniques: positive and negative cases; white, black, and gray box; boundary value analysis and equivalence partitioning; use case testing; root cause analysis.
- Clear written and verbal communication. Gives specific technical answers, not generalities. Says plainly what they do not know. Writes short daily updates in Jira and writes their own user stories and tasks. Comfortable on voice and video calls with the team.
- Takes ownership and initiative without hand-holding and gets ahead of problems. Emotionally stable, honest, and professional, especially when troubleshooting under pressure.
- Can explain how they would pick testing tools and approaches for a given system, not only how to run the tools they already know.
- Any certifications listed on the resume must be current and verifiable.
- Keeps up with senior engineers and architects. Learns new tools quickly.
- Willing to earn the AWS Solutions Architect Associate certification within the first year if not already held.
Clearance Required: Public Trust.
Preferred Skills and Experience
- AWS certification (Solutions Architect Associate or higher) strongly preferred.
- Front-end development experience (JavaScript, single-page applications) or AWS engineering experience. Either one is very helpful. Both are not required.
- Statistics for testing non-deterministic systems: sampling design, sample size, and confidence intervals; precision, recall, F1, and confusion matrices; distributions that are not bell curves; drift detection.
- Generative AI, IDP, and OCR evaluation experience: building ground truth, eval harnesses, versioned prompts, and hallucination detection. Knows the difference between prompt engineering, model tuning, and model training, and how to protect PHI/PII under each.
- Uses AI coding assistants such as Claude Code or Copilot to write test automation and to generate and expand test scenarios.
- Transformer fundamentals at a whiteboard level: tokenization, embeddings, self-attention, next-token prediction, and sampled decoding. Can use that to explain why outputs vary between runs, why long documents get worse results, and why models invent values that look right.
- Automated security testing in a continuous deployment environment.
- Behavior Driven Development (BDD) testing; accessibility testing (WCAG 2.1, Section 508).
- CI tools (GitHub Actions, Jenkins, or similar); Linux scripting.
- Java or C#/.NET for maintaining legacy applications.
- Experience on VA or other federal programs
- Startup experience a plus.
Posted Salary Range
USD $62,000.00 - USD $96,000.00 /Yr.
Bachelor's with 2-5 years experience; U.S. citizen or permanent resident; reside in CONUS; pass background check; test automation with Cypress or Playwright; REST API testing; SQL and Python; AWS testing; Scrum; Jira, Xray, Confluence; Public Trust clearance.
Key Responsibilities
- building automated tests
- designing test cases
- performing root cause analysis
Skills & Tools
Cypress, Playwright, Selenium, Postman, SQL, Python, AWS, S3, CloudWatch, Jira, Xray, Confluence, Locust, PostgreSQL, GitHub Actions, Jenkins, Linux, Java, C#/.NET, Claude Code, Copilot
Job Details
- Category: Information Technology
- Seniority: Mid Level
- Commitment: Full Time
- Workplace: Remote — United States
- Salary: USD 62,000 – 96,000 / year
- Languages: English
About GovCIO
A technology company building and modernizing federal digital services on AWS GovCloud. — Industry: Information Technology