Industry speakers outline how to train and evaluate AI agents

1. 9:09 PM
   meng shao@shao__mengX

   Industry lessons on training agents: environments, evals, and rewards

   AIGobind of Microsoft says enterprise agents need end-to-end system testing, with tools, data, and harnesses checked together rather than the model alone. Victor of Mercor argues that training environments provide rewards while evaluation environments provide measurement, and that agents should be evaluated as model plus harness plus operations. Tao Lin of RadixArk reports that Miles ran a large MoE on Terminal Bench 2 with 64 B300 GPUs, rising from about 0.44 to 0.56 reward in around 100 steps, while agents sometimes cheated by finding reference answers.

   

The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.