UC Berkeley paper finds research agents need periodic forgetting to keep improving

AISummary

A UC Berkeley paper, Scaling Discovery Requires Forgetting, reports that research agents that keep their full history become anchored on past attempts and stop improving.

The authors periodically discard each agent's session and candidate history and restart from the best candidate found so far.

The post reports gains of up to 15% over parallel agents and up to 39% over a single agent with the same budget across heuristic programming, kernel optimization, and model training tasks.

Post on XView on X

elvis

@omarsar0

Brilliant paper on scaling research agents from UC Berkeley.

(bookmark it)

They propose an unexpected and interesting way to scale automated research with agents.

It's common to see research agents keep their full history and iterate on it to solve a task. After a while, each agent anchors on its own past attempts, makes smaller edits, and extra budget stops improving things.

So this work induces forgetting in the agents.

Periodically, they discard each agent's session and the candidate history, and then restart from the best candidate found so far.

Across heuristic programming, kernel optimization, and model training tasks, this gives up to 15% over parallel agents and up to 39% over a single agent with the same budget.

https://arxiv.org/abs/2610.20563

Chat with Paper: https://academy.dair.ai/papers/scaling-discovery-requires-forgetting-2610.20563

Source: elvis · x.comPublished Oct 15, 9:50 PM ET

UC Berkeley paper finds research agents need periodic forgetting to keep improving

Topics