Note: The job is a remote job and is open to candidates in USA. VetsEZ is seeking a senior engineering professional to support a federal government healthcare modernization project. The role focuses on testing and automating an AI-powered patient health data summarization capability within the Joint Longitudinal Viewer Clinical Decision Support platform, including validating healthcare data, AI-generated outputs, interoperability workflows, and quality controls.
Responsibilities
- Develop comprehensive test plans, test cases, and acceptance criteria for AI-powered patient summarization capabilities
- Create traceability between business requirements, clinical data standards, and test scenarios
- Design test coverage across clinical document types, including Continuity of Care Documents (CCD), Discharge Summaries, Progress Notes, and History & Physical documents
- Develop positive, negative, boundary, and edge-case testing scenarios for AI-enabled clinical applications
- Analyze and validate C-CDA XML documents, including headers, sections, templates, and clinical entries
- Verify accurate extraction and summarization of clinical information, including medications, allergies, laboratory results, procedures, care plans, and diagnoses
- Ensure AI-generated summaries accurately reflect source clinical documentation and maintain traceability to original clinical sources
- Perform functional, integration, system, regression, performance, and user acceptance testing
- Develop and maintain automated test suites for document ingestion, AI summarization, APIs, and user interfaces
- Test CDS workflows and patient-context-driven launch scenarios across multiple data sources and patient encounters
- Support CI/CD pipelines through automated quality gates and continuous testing
- Build automated validation frameworks for AI-generated summaries
- Evaluate AI-generated content for clinical completeness, consistency, usability, accuracy, and reliability
- Identify omissions, inaccuracies, hallucinations, and other AI-generated defects that could impact clinical workflows
- Validate AI guardrails, monitoring capabilities, auditability, and quality controls
- Participate in defect triage, root-cause analysis, and release-readiness activities
- Create test documentation, defect reports, traceability matrices, and quality assessments
- Collaborate with software engineers, solution architects, clinicians, product owners, cybersecurity teams, and government stakeholders
- Leverage AI-assisted tools to accelerate test development, test automation, regression testing, defect analysis, and quality assurance activities
- Take on additional tasks and responsibilities as needed to support team objectives and ensure the success of the project
Skills
- The candidate must reside within the continental US
- * Bachelor's degree in Computer Science, Information Technology, Health Informatics, Engineering, or a related technical discipline, or equivalent experience
- * Minimum of 5 years of experience testing healthcare software applications
- * Strong knowledge of Consolidated Clinical Document Architecture (C-CDA) standards, including clinical document structure, sections, templates, and entries
- * Hands-on experience reading, validating, and troubleshooting XML-based healthcare data
- * Experience with healthcare interoperability standards and clinical data exchange
- * Experience testing Clinical Decision Support (CDS) applications and workflows
- * Familiarity with SMART on FHIR, CDS Hooks, REST APIs, and related interoperability technologies
- * Experience with test automation frameworks, API testing tools, and defect management platforms
- * Experience developing and maintaining automated testing solutions within Agile and DevSecOps environments
- * Experience with AI-assisted testing tools and techniques for test case generation, automation development, regression testing, defect analysis, and quality assurance
- * Familiarity with Generative AI and Large Language Models (LLMs), including using AI to create and maintain automated test scripts, validation frameworks, and test data
- * Experience validating AI-generated outputs and establishing quality controls for accuracy, consistency, repeatability, and reliability
- * Strong analytical, troubleshooting, and problem-solving skills
- * Excellent written and verbal communication skills
- U.S. Citizenship is required
- Ability to obtain and maintain a Government/Public Trust clearance, including required fingerprinting, when required for the position
- * Experience supporting Department of Veterans Affairs (VA) healthcare systems, including JLV or similar clinical viewer applications
- * Knowledge of USCDI and healthcare interoperability requirements
- * Familiarity with healthcare terminologies such as SNOMED CT, LOINC, RxNorm, and ICD
- * Experience validating AI/ML-enabled healthcare solutions
- * Experience with AI-powered testing platforms such as GitHub Copilot, Microsoft Copilot, Testim, Functionize, Mabl, or similar tools
- * Experience creating autonomous or semi-autonomous AI testing workflows, including test-case generation, coverage analysis, synthetic test data, and automated defect triage
- * Understanding of prompt engineering and AI evaluation methodologies for software quality assurance
- * Knowledge of accessibility and usability testing within healthcare environments
- * Experience working within Agile, DevSecOps, and cloud-native development environments
- * Exposure to healthcare data analytics and modernization initiatives within federal healthcare agencies
Benefits
- Medical/Dental/Vision.
- 401k with Employer Match.
- PTO + Federal Holidays.
- Corporate Laptop.
- Training Opportunities.
- Remote Opportunity.
Company Overview