Brilliant paper on scaling laws for agentic RL data.
It's a great paper worth reading if you are synthesizing data for agentic RL.
(bookmark it)
They fit scaling laws for agentic RL data along three axes: tasks, environments, and trajectories. Across model sizes, families, domains, and both PPO and GRPO, the exponents order the same way, with environments above tasks and tasks above trajectories.
The environment advantage seems to come mainly from world diversity, meaning genuinely different settings. Varying the structure of the same world helps about as much as adding tasks.
The laws also give each synthesis method a discount factor and a synthetic-to-real ratio at which training starts to collapse. Those ratios differ by an order of magnitude across axes.
In addition, allocating the budget with their recipe gives up to 1.7x compute efficiency over the common practice of generating more trajectories.
https://arxiv.org/abs/2610.23946
Chat with Paper: https://academy.dair.ai/papers/how-data-scales-in-agentic-reinforcement-learning-laws-and-synthesis-strategies-2610.23946