vLLM and NVIDIA make RL rollout and training logprobs bitwise identical on DeepSeek-V4-Flash

1. 10:50 AM
   vLLM@vllm_projectOfficial

   vLLM releases an open-source agent skill for debugging model diffs

   AIvLLM says it distilled its workflow for finding and fixing numerical divergence into an open-source agent skill, pull request 61551 on GitHub. The workflow finds the first diverging op, builds a minimal repro, fixes it, re-verifies, then profiles and optimizes. The post says DeepSeek-V4.1 has reached zero diff, with a follow-up coming, and that end-to-end training wall-clock speedup remains to be measured.

   
2. 10:50 AM
   vLLM@vllm_projectOfficial

   vLLM reports zero train-inference KL on Nemotron-3.5-Lightning-30B-A3B

   AIvLLM says its method keeps train-inference KL at exactly 0 on Nemotron-3.5-Lightning-30B-A3B, a Mamba-2 hybrid with NVFP4, in a filtered SWE-bench Pro training subset. Training reward rises from 0.56 to 0.59, while the baseline run falls from 0.56 to 0.22, comparing the first and last 10 steps. Rollout throughput reaches 87.5% of baseline over the first 30 steps, and the post notes these are training results, not held-out benchmark scores.

   
3. 10:50 AM
   vLLM@vllm_projectOfficial

   vLLM reports zero logprob diff for DeepSeek-V4-Flash RL on 32 GB200 GPUs

   AIvLLM says DeepSeek-V4-Flash showed zero logprob difference at every step over 100 RL steps on 32 GB200 GPUs. Rollout throughput averaged 24.22 output tokens/ms, 91.3% of the 26.53 baseline, and 88.6% over the first 89 steps. The post notes that response lengths differed across runs, so the figures are rollout-run measurements rather than an isolated engine benchmark.

   
4. 10:50 AM
   vLLM@vllm_projectOfficial

   vLLM makes RL rollout and training bitwise identical for faster training

   AIvLLM says same weights do not guarantee the same policy, because inference optimizations can change the logprobs that training recomputes. The project says it makes rollout and training bitwise identical while keeping high-performance inference paths. The post is a thread that links to a vLLM blog article.

   

1. vLLM BlogOfficialPick

   vLLM blog: bitwise-identical rollout and training logprobs stabilize RL training

   AIThe vLLM blog reports that training and inference compute bitwise-identical logprobs for every sampled token, giving a train-inference KL of 0 at every step. On DeepSeek-V4-Flash, a high-performance baseline's KL rose from 2e-3 to 29.9 while its reward fell back. The consistent version kept 91.3% of baseline rollout throughput, and the authors packaged the process as an open-source agent skill that they also applied to Nemotron-3.5.

The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.