AI auditors catch most agent misbehavior but flood reviewers with false alarms
AIAndy Hall, a researcher at the Anthropic Institute, replicated METR's MALT study to test whether AI models can flag agent misbehavior that human reviewers found. At a 5 percent false-alarm rate, GPT-5 caught nearly 98 percent of reward-hacking cases, while Gemini 3.8 Flash found barely half.
Source details
The Decoder · Manuel Uth · the-decoder.comPublished · added here
Original title: AI auditors catch most agent misbehavior but flood reviewers with false alarms, new study finds