Congrats to @guohao_li and @env_lab_ai on AgenticRL-Bench!
Glad Miles helped power this effort. Love seeing real RL challenges turned into benchmarks that the agent community can build on. Excited to see what comes next!
Eigent AI introduces AgenticRL-Bench, a benchmark for agents doing agentic RL
AIEigent AI, through The Environment Lab, introduced AgenticRL-Bench with 22 tasks rebuilt from real RL incidents in systems including Miles, Slime, Verl, SGLang and Harbor. On the 19 verifiable tasks, none of the six agents passed more than 14.0% of attempts.
@radixark
Can agents perform agentic RL? If RSI happens, agents will need to take over researchers' work on agentic RL Introducing AgenticRL-Bench: the first release from The Environment Lab @env_lab_ai effort at Eigent AI 22 tasks rebuilt from real-world RL incidents on production-grade agentic RL systems, including Miles, Slime, Verl, SGLang, and Harbor, spanning 8 areas of the stack: training data, harnesses, algorithms, RL infrastructure, recipes, evaluation, monitoring, and capability transfer We ran 6 agents with 3 attempts per task. On the 19 verifiable tasks, no agent passes more than 14.0% of its attempts Across the three open-ended tasks, agents improved real policies or training steps using 4×H200 GPUs over 10–16 hours. On one of these tasks, Zero Signal Hill Climb, where the base model fails to solve any of the 38 tasks, the best attempt developed its own method: the agent first used hinted text feedback to generate successful rollouts, then removed the feedback from the trajectories and blended these successful rollouts with regular GRPO rollouts to form mixed training groups. It monitored the training process and adjusted KL loss weight to prevent training collapse. This improved pass@1 from 0 to 0.186 (12 out of 38 tasks)
Source details
RadixArk · x.comPublished · added here
Original title: Congrats to @guohao_li and @env_lab_ai on AgenticRL-Bench!