These guys tested an auto-tuner that optimizes the full reinforcement learning loop.
Across eight math and coding jobs:
• Training steps became faster by 1.42x to 4.93x (3.5x median).
• They got one DeepSeek setup to run faster on 40 GPUs than an earlier setup did on 56.
• In one coding job, they cut each training step from 10.7 to 7.9 minutes.
Up to 5× faster RL training steps, on the same GPU budgets. Our internal auto-tuner optimizes the full RL loop: training, inference, and environment execution. It profiles each job, forecasts promising configuration changes, tests them in real runs, and follows the bottleneck as it moves. Across 210 runs on NVIDIA B300 GPUs, we tested four dense and MoE models on math and agentic coding workloads. All eight jobs improved: 1.42-4.93x faster steps, with a median speed-up of about 3.5x. One revealing result: nearly doubling inference-engine throughput in a benchmark didn’t shorten the RL step. Sandbox execution absorbed the gain. That’s why we measure the complete job, alongside numerical checks and longer reward comparisons. GPU counts stayed fixed; some configurations added sandbox hosts. We’ve shared the system, results, and five engineering lessons, including what didn’t work, in the article below:
Santiago · x.comPublished · added here
Original title: These guys tested an auto-tuner that optimizes the full reinforcement learning loop.
