Note: The job is a remote job and is open to candidates in USA. Accuity partners with hospitals and health systems through a technology-enabled, physician-led model that improves clinical documentation integrity, coding accuracy, reimbursement optimization, and quality outcomes. The Data Scientist develops, evaluates, and monitors predictive and generative AI systems supporting clinical documentation and revenue cycle workflows, with responsibility for ground truth datasets, evaluation frameworks, model development, governance, and production readiness.
Responsibilities
- Develop, train, and improve predictive and machine learning models supporting chart triage, prioritization, documentation integrity, and revenue cycle outcomes
- Maintain and improve models already running in production, including retraining cadence, feature review, threshold tuning, and recalibration as data and workflow change
- Identify and correct sampling and selection bias in training data, including bias introduced when a model’s own decisions determine which records are subsequently observed
- Contribute to applied work on large language model and agent-based systems, including prompt and workflow design, tool integration, and systematic failure mode analysis
- Build feedback loops so findings from clinical audit and production review flow back into model, threshold, and system decisions rather than stopping at a report
- Prototype and test new approaches, and state plainly and early when an approach does not work
- Build and maintain ground truth datasets, including sourcing, annotation coordination with clinical subject matter experts, quality assessment, and remediation of known data defects
- Design evaluation samples deliberately, understanding when a stratified or weighted sample is appropriate, when a near-production distribution is required, and when results across runs are not comparable
- Define, compute, and document the metrics that determine production readiness, and ensure another person can reproduce them from source data
- Analyze performance by clinically meaningful segment, including diagnosis group, service line, payer, and case complexity, rather than reporting aggregate accuracy alone
- Analyze error in both directions, distinguishing false positives from false negatives and quantifying the clinical and financial consequence of each
- Maintain evaluation tooling and harnesses so experiments are repeatable and every result is traceable to a specific dataset, configuration, and version
- Monitor deployed models and AI systems for accuracy, calibration, drift, and subgroup performance, and surface issues before they appear as business impact
- Maintain documentation of model design, assumptions, limitations, training data, and known failure modes to a standard that supports audit and clinical review
- Support AI governance requirements, including human-in-the-loop design, decision traceability, and evidence of validation
- Contribute to the definition and computation of metrics that carry commercial or contractual weight, ensuring they are reproducible and auditable
- Work directly with physicians, coders, and clinical subject matter experts to define correct outcomes, resolve disagreements about ground truth, and validate model behavior against real clinical judgment
- Translate clinical and operational problems into well-posed analytical problems, and translate results back into terms a clinical or business audience can act on
- Present findings, limitations, and recommendations honestly, including where results are inconclusive or where the available data will not support the question being asked
- Help the business understand what a model can and cannot be relied on to do
- Work alongside external AI development partners, independently reproducing and verifying reported results rather than accepting them as delivered
- Review partner methodology, sampling design, and metric definitions, and raise discrepancies clearly and early
- Absorb knowledge from partner-delivered systems so Accuity can evaluate, maintain, and improve them without ongoing external dependency
- Partner with AI Engineering to move validated models and systems into production, providing the specifications, evaluation criteria, and acceptance thresholds that define success
- Partner with Data Architecture on the data assets, structure, quality, and access required for model development and evaluation
- Write clean, reviewable, version-controlled code and work in shared repositories rather than personal notebooks
- Document work so another data scientist can reproduce it without the original author present
- Handle protected health information in accordance with HIPAA, HITRUST, and Accuity policy at all times
- Follow approved practice for data access, data movement, model routing, and any use of third-party services involving PHI
- Support audit readiness by producing accurate, timely evidence of validation and evaluation work
- Perform additional responsibilities as needed to support data science, technology, and organizational goals
Skills
- • Bachelor's degree in computer science, statistics, mathematics, engineering, data science, or a related quantitative field required
- • 4+ years of applied data science, machine learning, or applied AI experience, including work that reached production
- • Demonstrated experience building and evaluating models that were actually used to make decisions, rather than research or coursework alone
- • Experience with evaluation methodology, including ground truth construction, sampling design, and metric definition
- • Strong proficiency in Python and SQL
- • Healthcare data experience required; experience with clinical coding, DRG assignment, CDI, or claims and revenue cycle data strongly preferred
- • Evaluation Rigor: Measures with methods that are reproducible, auditable, and honest about what they do and do not prove
- • Data Skepticism: Assumes a dataset has defects until checked, and finds contaminated labels and bad ground truth before they shape a conclusion
- • Scientific Honesty: Reports negative and inconclusive results as readily as positive ones, and does not let a preferred conclusion select the analysis
- • Statistical Judgment: Chooses appropriate samples, methods, and metrics for the question, and knows when a result is not comparable to another
- • Analytical Depth: Looks past aggregate numbers to segment, subgroup, and error-pattern behavior where the real story usually sits
- • Clinical Curiosity: Engages seriously with the clinical and coding domain rather than treating it as a source of features
- • Applied Orientation: Works toward a decision that gets made or a system that gets used, rather than analysis for its own sake
- • Communication: Explains method, result, uncertainty, and limitation clearly to technical, clinical, and business audiences
- • Reproducibility: Produces work another person can rerun and reach the same answer, with code, data, and configuration under version control
- • Collaboration: Works effectively with clinicians, engineers, and external partners, including when the disagreement is technical
- • Adaptability: Moves between classical modeling, applied AI, and data quality work as the problem requires
- • Remote-Work Effectiveness: Communicates proactively, documents decisions, and maintains visibility into work in a fully remote organization
- 1) Physical Requirements: The requirements described here are representative of those that must be met by an employee to successfully perform the essential functions of this job with or without reasonable accommodations. Unless otherwise indicated, Accuity positions require interaction with people and technology while either sitting or standing. Employees must be able to communicate via phone, email, etc. and sit for extended periods of time, with or without reasonable accommodations. Physical effort and exposure to physical risk are limited to that of an office role / environment
- • Master's degree in a related field preferred
- • Exposure to large language model or agent-based systems, including prompt design and systematic evaluation of non-deterministic output, preferred
- • Experience working with protected health information in a HIPAA-regulated environment preferred
- • Experience with cloud data and machine learning platforms, preferably Microsoft Azure and Databricks, preferred
Benefits
- Remote work
Company Overview
Company H1B Sponsorship