Note: The job is a remote job and is open to candidates in USA. b.well Connected Health is developing Bailey, a clinical AI system that orchestrates LLMs, manages health safety evaluation, and integrates with clinical data. The Senior ML/Platform Engineer will own the path from ML prototype to production by building deployment pipelines, evaluation frameworks, monitoring systems, and observability infrastructure, while hardening services and collaborating with data scientists.
Responsibilities
- Deploy and version ML artifacts that currently move to production ad hoc
- Stand up observability for the existing LangGraph agent orchestration (what's happening, how often, how well)
- Instrument the health safety evaluator so its 5-dimension scoring is monitored continuously, not checked manually
- Build the evaluation infrastructure described in our product strategy: drift detection, distributional monitoring, safety-weighted quality scoring
- Own CI/CD for ML artifacts and agent configurations
- Harden existing services (conversation persistence, MCP tool orchestration, clinical skills framework) for production reliability
- Get up to speed on Bailey's evaluation documentation and use it to shape how we test and monitor Bailey's behavior going forward, including flagging where traditional ML methods (not just LLM-based approaches) could deliver the same outcome more efficiently
- Be the person who knows whether the system is healthy, degrading, or broken before users or clinicians notice
- Collaborate with data scientists so their work reaches production without a handoff gap
- Keep testing and monitoring aligned with Bailey's evolving evaluation documentation, and keep surfacing opportunities to swap in traditional ML where it's more efficient than an LLM-based approach
- Evolve the platform as the product grows (voice, memory, multimodal are on the roadmap)
Skills
- * You've built production ML infrastructure. Not just used it. Deployment pipelines, serving systems, model tracking/registries (MLflow or equivalent), monitoring that pages you at 2am. You know the difference between a model that works in a notebook and one that works in production
- * You're a strong software engineer first. Distributed systems, containerization, event-driven architectures, CI/CD. ML is the domain; engineering is the craft
- * You're comfortable owning something end-to-end. No one will hand you a spec and check your work weekly. You'll identify what needs to exist, propose how to build it, and ship it
- * You work well in small teams. You communicate clearly, you surface problems early, and you're comfortable with the visibility and accountability that comes with a four-person team
- * Experience in healthcare or regulated industries (you understand why audit trails and compliance aren't afterthoughts)
- * Background in ML research or applied ML (you can have informed opinions about model behavior, not just model infrastructure)
- * Familiarity with AWS ML services like Bedrock and SageMaker, which are part of our current stack
- * Experience with LLM orchestration frameworks (LangChain, LangGraph, or similar)
- * Exposure to clinical data standards (FHIR, HL7) or health information systems
- * Publications, open-source contributions, or side projects that show you go deeper than your day job requires
Benefits
- Stock options
- Incentive pay for eligible roles
Company Overview