vLLM v0.32.0 adds structured decision endpoint and low-precision KV support

AISummary

vLLM released v0.32.0 with 814 commits from 363 contributors, including 121 first-time contributors. The release adds a /v1/systemone structured-decision endpoint, NVFP4 and 4-bit KV cache support on several hardware platforms, and faster startup with multi-API workers reporting +43–85% req/s.

Post on XView on X

vLLM

@vllm_project

🚀 vLLM v0.32.0 is out! 814 commits · 363 contributors · 121 first-timers

🧭 New /v1/systemone structured-decision endpoint + generic hf tool/reasoning parser
🗜️ Low-precision KV everywhere: NVFP4 on SM8x/90/12x, FP8 KV default for GLM-5.x, 4-bit KV on ROCm
🛠️ Model Runner V2: GPU n-gram speculator, Elastic EP, PP+CP for MLA, now default on ROCm
⚡ Faster startup & serving: precompiled kernel downloads, autotune preload, +43–85% req/s with multi-API workers
🆕 EmbeddingGemma 2, MiniCPM-V 4.7, Nemotron 3.5 ASR
🔒 3 security fixes + CORS hardening

Full notes → http://github.com/vllm-project/vllm/releases/tag/v0.32.0

Source: vLLM · x.comPublished Oct 18, 9:40 AM ET · added here 9:48 AM ET

vLLM v0.32.0 adds structured decision endpoint and low-precision KV support

Topics