vLLM Adds Agent-Assisted CuTe DSL Decode Path for Kimi K3 on NVIDIA GB300
Overview
vLLM reports a new Kimi K3 decode path built with agent-assisted CuTe DSL kernels for NVIDIA GB300 GPUs, covering attention, MoE and communication across 8 or 16 GPUs.
Benjamin Chislett of NVIDIA describes the work, and says Kimi K3 is the first contribution made with this approach.
vLLM says the path reaches 842 tokens/s at batch 1 on 16 GB300 GPUs with DSpark (synthetic AL=6), 1.73× stock vLLM. It also reports up to 1.8× higher P90 interactivity on the AgentX workload.
The post lists review principles for the agent-written code: isolating changes, bounding scope, validating correctness, and checking performance broadly.
Written by AI from the articles below · updated Oct 16, 9:34 PM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
- vLLM BlogOfficialPickvLLM explains how it optimized Kimi K3 decode with agent-assisted kernels
AIvLLM reports a new Kimi K3 decode path built with agent-assisted CuTe DSL kernels for NVIDIA GB300 GPUs. It reaches 842 tokens/s at batch 1 on 16 GB300 GPUs with DSpark (synthetic AL=6), 1.73× stock vLLM, and up to 1.8× higher P90 interactivity on the AgentX workload. The post also describes review principles of isolating changes, bounding scope, validating correctness, and checking performance broadly.
Heat trend
Not enough continuous observations to show a trend yet.