How Numan built an eval-first AI health coach with LangGraph and LangSmith
AISummary
Numan describes how it built Nu, an AI health coach, to avoid medical diagnosis, prescriptions, and clinical decisions.
The team found that single-turn fixtures and scripted conversations failed to test multi-turn coaching, so it scores per-category, runs the same graph in production and tests, and simulates patient conversations.
Clinicians sign off the rubric, and a boundary regression blocks releases.
AIWhy it matters
The post shows how a regulated health agent was tested with per-category evals, simulated patients, and clinician-owned rubrics, which transfers to other high-stakes chatbots.
Source: LangChain Blog · langchain.comPublished · added here