vLLM adds day-0 support for Kimi K3.1 Pro on NVIDIA, AMD, TPU and T-Head
AIvLLM says it supports Moonshot AI's Kimi K3.1 Pro on day 0 across NVIDIA GPUs, AMD GPUs, Google TPUs and Alibaba T-Head PPUs. The post says the model adds Sparse Attention (DSA) to all 24 full-attention layers, and that vLLM packs its MLA latent, indexer keys and KDA state into one KV cache. vLLM also reports up to 1.69× faster time to first token than head-sharded prefill at 340K tokens on 8× GB300.
AIWhy it matters.
The post explains the sparse attention design and KV cache layout behind day-0 serving, useful for engineers planning long-context deployments of large open-weight models.
Source details
vLLM Blog · Inferact and the vLLM Team · vllm.aiPublished · added here
Original title: vLLM Delivers Day-0 Support for Kimi K3.1 Pro