vLLM serves DeepSeek-V4.1-Flash at 954K tokens/s per GPU on GB300 NVL72
AIvLLM reports 954K tokens/s per GPU at a P90 of 196 tokens/s per user on one GB300 NVL72 rack, using P/D disaggregation and Wide-EP, which it says is 7.9 times its earlier single-node result. The post says vLLM decode throughput is within 6% of DeepInfer, DeepSeek's own engine, across batch sizes 8 to 256, and attributes the gains to prefill, EPLB and CPU-overhead fixes.
AIWhy it matters.
The post explains which measured bottlenecks were fixed and how each change moved throughput, useful for engineers tuning large MoE serving stacks.
Loading source posts…
Source details
vLLM Blog · Inferact and the vLLM Team · vllm.aiPublished · added here
Original title: Serving DeepSeek-V4.1-Flash at Scale: vLLM Matches DeepInfer and Scales to GB300 NVL72