Berkeley study finds self-evolving agents gain and lose tasks, hidden by averages

A paper from UC Berkeley and collaborators reports that self-evolving AI agents, which rewrite their own prompts, skills, tools, or code while the model stays frozen, lose tasks they previously solved even as their scores rise.

The authors tested four evolution methods, including GEPA and HyperAgents, using GPT-5.4 on two harnesses and two benchmarks, re-running each evolved agent eight times. All 16 evolved agents gained some tasks and lost others, and one raised reliability by 5.2 points while losing 10 previously covered tasks. The authors say average scores conceal these losses, as reported by Rohan Paul on X.