NVIDIA Research and Inferact present REINFORCE Zero for stable asynchronous RL
Overview
NVIDIA Research and Inferact present REINFORCE Zero, a recipe for stable asynchronous reinforcement learning with importance sampling, in a paper by Jian Hu et al.
The vLLM Blog, a first-party source, reports that aligning rollout and training kernels removes the fluctuating mismatch between them. The authors say this lets unclipped importance sampling correct the remaining weight lag and keeps training stable for 20,000 updates.
The authors also report 2.40× the training-step throughput of synchronous training with aligned kernels. These results come from the authors' own paper and have not been independently verified.
Written by AI from the articles below · updated Nov 6, 10:34 AM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
- vLLM BlogOfficialPickNVIDIA Research and Inferact present REINFORCE Zero for stable asynchronous RL
AINVIDIA Research and Inferact present REINFORCE Zero, a recipe for stable asynchronous reinforcement learning with importance sampling. The paper reports that aligning rollout and training kernels removes the fluctuating mismatch, letting unclipped importance sampling correct the remaining weight lag and keep training stable for 20,000 updates. The authors also report 2.40× the training-step throughput of synchronous training with aligned kernels.
Heat Trend
Not enough continuous observations to show a trend yet.