vLLM reports 842 tokens/s Kimi K3 decode on NVIDIA GB300 NVL72

AISummary

vLLM reports Kimi K3 decode reaching 842 tokens/s at batch 1 on NVIDIA GB300 NVL72 using DSpark. The unified CuTe DSL decode path fuses attention, MoE and communication, delivering 1.5–2× higher decode throughput in several tested configurations and up to 1.8× higher P90 interactivity on the SemiAnalysis AgentX workload.

AIWhy it matters

The post gives concrete throughput, interactivity and configuration results, showing how a unified decode kernel changes serving performance for a large model.

Post on XView on X

vLLM

@vllm_project

Kimi K3 decode on NVIDIA GB300 NVL72: 842 tokens/s at batch 1 with DSpark 🚀

A unified CuTe DSL decode path fuses attention, MoE and communication, delivering 1.5–2× higher decode throughput in several tested configurations and up to 1.8× higher P90 interactivity on the @SemiAnalysis_ AgentX workload.

The post covers the kernel work, the correctness and long-context evals behind it, and benchmark sweeps across batch sizes, context lengths and parallelism configurations.

Full write-up: http://vllm.ai/blog/2026-10-16-kimi-k3-megakernel

Great work and collaboration with the @NVIDIA team!

Source: vLLM · x.comPublished Oct 17, 9:55 PM ET · added here 10:05 PM ET

vLLM reports 842 tokens/s Kimi K3 decode on NVIDIA GB300 NVL72

Topics