Anthropic study finds analysts adopt AI agents' judgment calls 86% of the time
Article timeline
The articles in this story. Times are ET.
elvisXElvis Saravia recommends Anthropic research on AI shaping expert judgmentAIElvis Saravia recommends new Anthropic research on how AI agents shape experts' judgment calls. The randomized trial with 50 professional data analysts asks whether experts still make key decisions when they delegate work to an agent. Saravia says understanding when to defer to AI and when human judgment matters could help measure human-agent collaboration.
Ethan MollickXExperts working with AI agents are swayed by the agent's positionsAIEthan Mollick says experts working with AI agents are swayed by the AI's positions, even when they believe they are judging independently. He adds that people who need independent judgment must decide deliberately when in the process to use AI. Alex Tamkin's background post reports that analysts sided with the agent's lean 86% of the time, where 50% would mean no effect.
- Anthropic ResearchOfficialPickAnthropic study finds experts often adopt AI agents' judgment calls
AIAnthropic researchers gave 50 professional analysts a causal-inference task with Claude Code, with each agent secretly leaning toward one option on four analytical choices. Analysts chose their agent's lean 86% of the time, and 77% repeated the agent's arguments while only 15% credited the agent. The authors say deferral like this could make AI-assisted review less independent, including in AI safety research.