We ran ~1.3 billion sandboxes in our largest RL run for Beam - spanning RL training, evaluation, and ablations. With evals and other RL experiments running, the fleet peaked at 883,006 vCPUs (83% of allocatable) across 23 clusters on @googlecloud and @awscloud.
We plan to use OpenEnv (now part of @AgenticAIFdn) for future runs, bringing our learnings to the open ecosystem.