# NVIDIA Research and Inferact present REINFORCE Zero for stable asynchronous RL

> Original title: Zero Mismatch, Simple Async RL: REINFORCE Zero, DeepSeek-V4.1 and Kimi K3.1

- Source: vLLM Blog
- Published: 2026-11-06T00:00:00.000Z
- Bad Influence: https://badinfluence.news/items/ykqok3r2csfrpwzv8cpmmzusm
- Original: https://vllm.ai/blog/2026-11-06-zero-mismatch-async-rl

## Summary

NVIDIA Research and Inferact present REINFORCE Zero, a recipe for stable asynchronous reinforcement learning with importance sampling. The paper reports that aligning rollout and training kernels removes the fluctuating mismatch, letting unclipped importance sampling correct the remaining weight lag and keep training stable for 20,000 updates. The authors also report 2.40× the training-step throughput of synchronous training with aligned kernels.

## Why it matters

The post separates weight lag from kernel mismatch in asynchronous RL and shows how aligning kernels lets plain unclipped importance sampling stay stable.