LangChain describes eval-driven development for AI agents with a helpdesk tool example

LangChain's Divya Sanjeevan says eval-driven development starts by defining expected agent behavior, then builds evals and a baseline before comparing each change against it.

In an internal IT-helpdesk example, adding a ticket-creation tool raised the pass rate from 5 to 7 of 8 cases, but the agent also opened a ticket for a how-to question. Narrowing the tool description fixed all 8 cases, according to the report.