Applied AI Engineer

Applied AI Engineer

Department: Technology

Location: Remote

FLSA Status: Exempt

People Leader: No

Travel: Less than 10%

Summary

Company Summary

Accuity partners with hospitals and health systems through a technology-enabled, physician-led model that improves clinical documentation integrity, coding accuracy, reimbursement optimization, and quality outcomes.

Job Summary

The Applied AI Engineer designs, prototypes, and builds production applications and features that apply existing AI models and services to clinical documentation and revenue cycle workflows. Reporting to the Vice President, Data Science and AI, this role owns the application layer, including user experience, service integration, prompt design, safeguards, testing, and production performance.

The engineer partners with AI Engineering, Data Science, Product, Design, and software engineering teams to turn validated AI capabilities into reliable tools that clinicians, clinical coders, and revenue cycle teams can use in their daily work. This includes embedded features within existing products and standalone internal applications.

Success in this role is measured by the ability to deliver reliable, user-centered AI applications that produce measurable improvements in the effectiveness, efficiency, and quality of clinical, coding, and revenue cycle workflows across the Accuity ecosystem.

Responsibilities

AI Application Development

• Design, build, test, and maintain applications, features, and internal tools that use AI models, services, and agents provided by internal teams or approved vendors.
• Translate user and business problems into defined AI application requirements, including expected behavior, prohibited behavior, failure handling, and human review.
• Build embedded features within existing products and standalone applications when a dedicated tool better serves the workflow.
• Integrate AI services into application interfaces, workflows, and data systems without duplicating existing platforms or infrastructure.
• Develop prototypes to determine whether an AI capability solves the intended problem before committing to full production development.
• Design clear interfaces between applications and AI services, including request formats, responses, error handling, latency requirements, and usage constraints.
• Write clean, secure, maintainable, and reviewable production code.
• Participate in code reviews, technical design discussions, testing, and release activities.

Prompt Interaction and Experience Design

• Design and refine prompts, examples, structured outputs, and interaction patterns for specific application use cases.
• Design how AI-generated content, supporting evidence, citations, uncertainty, and limitations are presented to users.
• Provide users with appropriate review, editing, confirmation, and recovery options when AI output is incomplete or incorrect.
• Partner with Product and Design to create interaction patterns suited to AI-assisted clinical and operational work.
• Determine when an AI feature can act automatically, when it should request confirmation, and when it should defer to a person.
• Build feedback mechanisms that capture user corrections, overrides, and other signals for product and model improvement.
• Apply accessibility and usability standards to AI-powered interfaces.

AI Evaluation and Application Quality

• Define measurable product-level acceptance criteria for AI-powered features.
• Build repeatable tests that evaluate application behavior across expected, unusual, and high-risk scenarios.
• Create and maintain regression tests for prompts, service integrations, and model or configuration changes.
• Evaluate the quality of generated content, retrieval results, citations, structured outputs, and tool calls.
• Test application behavior when AI services return incorrect, incomplete, delayed, or unavailable responses.
• Analyze common failure patterns and work with AI Engineering and Data Science to resolve or mitigate them.
• Document known limitations and ensure that product behavior reflects those limitations.
• Escalate when a product requirement exceeds what an available model or service can reliably support.

Software Engineering and Production Ownership

• Own AI-powered application features from technical design through production release and ongoing maintenance.
• Develop automated tests appropriate to the application and its AI integrations.
• Follow established source control, code review, continuous integration, and deployment practices.
• Implement logging, telemetry, and monitoring to understand application performance and user behavior.
• Monitor production reliability, latency, usage, and cost.
• Investigate and resolve production issues involving application logic, integrations, prompts, or AI service behavior.
• Maintain technical documentation covering architecture, dependencies, configuration, support procedures, and known limitations.
• Work with platform and engineering teams to meet security, performance, availability, and support requirements.

Vendor and Tool Evaluation

• Evaluate third-party AI products, APIs, and tools against defined product requirements, cost, reliability, security, and data-handling constraints.
• Run structured trials using representative workflows and documented evaluation criteria.
• Compare vendor claims with observed performance and document material limitations.
• Recommend whether Accuity should build, buy, or integrate a capability in partnership with AI Engineering and Product.
• Monitor developments in relevant AI tools and services and identify capabilities that may address established product needs.
• Support the technical integration and ongoing evaluation of selected vendor products.

Security, Compliance, and Responsible AI

• Build AI-powered applications in accordance with HIPAA, HITRUST, PHI protection requirements, and Accuity policy.
• Use only approved models, services, vendors, data routes, and technical environments.
• Follow required controls and contractual requirements when applications process protected health information.
• Preserve human review and decision traceability when clinical, coding, or financial judgment is involved.
• Design safeguards against inappropriate data disclosure, unauthorized access, unsafe inputs, prompt injection, and unintended system actions.
• Communicate clearly what an AI feature does, what evidence supports its output, and where its limitations apply.
• Escalate proposed features or technical designs that could place protected health information, patient safety, or clinical decision quality at risk.
• Produce accurate documentation and evidence required for security, compliance, and audit review.

Collaboration and Ways of Working

• Partner with AI Engineering to consume production AI services and communicate application requirements.
• Partner with Data Science to understand model capabilities, evaluation results, and limitations for specific use cases.
• Partner with Product and Design to determine where AI can improve a workflow and where another approach is more appropriate.
• Partner with software engineering teams on application architecture, interfaces, performance, reliability, and deployment.
• Work with clinicians, coders, and operational subject matter experts to understand workflows and validate application behavior.
• Communicate tradeoffs involving performance, cost, accuracy, usability, and risk to technical, clinical, and business audiences.
• Document technical decisions, requirements, risks, and unresolved questions.
• Contribute to shared engineering standards and reusable application patterns.

Other Duties as Assigned

• Perform additional responsibilities as needed to support data science, technology, and organizational goals.

Qualifications

Education and Credentials

• Bachelor’s degree in computer science, software engineering, human-computer interaction, or a related technical field, or equivalent practical experience, required.
• Relevant advanced education, professional training, or technical certifications preferred.

Experience

• Required Experience:
  • Three or more years of software engineering, application engineering, or product engineering experience, including responsibility for production applications or user-facing features.
  • Experience building applications or features using large language models or other AI services through APIs.
  • Experience designing and refining prompts for defined application use cases.
  • Experience integrating internal or third-party AI and machine learning services into existing applications.
  • Experience addressing service latency, cost, reliability, error handling, and graceful failure.
  • Working knowledge of model capabilities, limitations, prompting, retrieval, and AI-assisted application design.
  • Experience implementing explainability and observability for AI-powered applications, including traceable outputs, source citations, structured logging, and monitoring that help users and support teams understand system behavior.
  • Healthcare data experience; familiarity with clinical coding, CDI, claims, or revenue cycle data.
• Preferred Experience:
  • Experience evaluating AI tools, APIs, or vendor products against technical and product requirements.
  • Experience contributing to build, buy, or integration decisions.
  • Experience partnering closely with product management and design on user-facing features.
  • Experience designing user interactions for AI-generated content, recommendations, or automated actions.

Core Competencies

• AI Product Judgment: Identifies where AI can improve a workflow and where it would introduce unnecessary risk or complexity.
• Application Engineering: Builds maintainable, secure, and reliable production applications.
• AI Integration: Integrates models and AI services into existing products and workflows through clearly defined interfaces.
• Evaluation: Uses repeatable tests and representative scenarios to determine whether an AI feature is ready for production.
• User Experience: Presents AI-generated output in a way that supports informed review and appropriate action.
• Production Reliability: Anticipates failures and designs effective monitoring, fallback, and recovery behavior.
• Security and Compliance: Treats data protection, access control, traceability, and approved routing as core design requirements.
• Technical Judgment: Selects an approach that meets the requirement without introducing unnecessary complexity.
• Communication: Explains application behavior, technical tradeoffs, uncertainty, and limitations clearly.
• Collaboration: Works effectively with engineering, data science, product, design, clinical, and business teams.
• Documentation: Records technical decisions, application behavior, dependencies, and limitations so others can support the work.
• Adaptability: Evaluates new tools and methods without adopting them before their value and risks are understood.
• Remote Work Effectiveness: Communicates proactively, documents decisions, and maintains visibility into work in a fully remote organization.

Additional Requirements

1) Physical Requirements: The requirements described here are representative of those that must be met by an employee to successfully perform the essential functions of this job with or without reasonable accommodations. Unless otherwise indicated, Accuity positions require interaction with people and technology while either sitting or standing. Employees must be able to communicate via phone, email, etc. and sit for extended periods of time, with or without reasonable accommodations. Physical effort and exposure to physical risk are limited to that of an office role / environment.

2) Position and Employment Statement: While this job description is intended to be an accurate reflection of the job requirements, management reserves the right to modify, add or remove duties from a job and to assign other duties as necessary and at any time. All positions at Accuity Delivery Systems, LLC, are at-will employment, and a position description is not a guarantee of a job or of job responsibilities.

 

While this job description is intended to be an accurate reflection of the job requirements, management reserves the right to modify, add or remove duties from a job and to assign other duties as necessary and at any time. All positions at Accuity are at-will employment, and a position description is not a guarantee of a job or of job responsibilities. 

Bachelor's degree in computer science or related field or equivalent experience; 3+ years software engineering experience; experience with LLMs and AI APIs; healthcare data experience; HIPAA/HITRUST compliance knowledge.

Key Responsibilities

• building AI applications
• integrating AI services
• evaluating AI quality

Job Details

• Category: Software Development
• Seniority: Mid Level
• Commitment: Full Time
• Workplace: Remote — United States
• Languages: English

About Accuity

A technology-enabled, physician-led company improving clinical documentation integrity, coding accuracy, and reimbursement optimization. — Industry: Healthcare