vLLM adds day-zero serving support for Kimi K3.1 Pro on NVIDIA, AMD, TPU and T-Head hardware
Article timeline
The articles in this story. Times are ET.
vLLMOfficialvLLM adds day-zero support for Kimi K3.1 Pro on NVIDIA, AMD, TPU and Zhenwu hardwareAIvLLM says it has day-zero platform support for Kimi K3.1 Pro across NVIDIA GPUs including Vera Rubin, AMD Instinct MI355X, Google Cloud TPUs, and Zhenwu 810e and M890 chips. The post lists launch images for NVIDIA CUDA 13.0 and 13.4 and AMD ROCm, and links to Moonshot AI's model evaluations and vLLM's Day-0 technical blog.
- vLLMOfficialvLLM explains K3.1 attention and serving architecture
AIvLLM says K3.1 combines DSA sparse attention with recurrent Kimi Delta Attention, storing MLA latents, indexer keys and KDA state in one packed cache. The serving work adds token-sharded prefill, exact top-k merging for decode context parallelism, and a fused low-latency decode path.
InferactOfficialInferact expands Moonshot AI partnership to serve Kimi K3.1 Pro on vLLMAIInferact says it has expanded its partnership with Moonshot AI to bring the Kimi model series to production with vLLM. Kimi K3.1 Pro is available on the Inferact Platform from launch and can be served in vLLM now. Inferact says it tunes and validates the serving path with Moonshot before each release and upstreams its production learnings to vLLM.
- vLLMOfficialvLLM adds day-zero support for Kimi K3.1 Pro on NVIDIA, AMD and TPU hardware
AIvLLM says it provides day-0 platform support for Kimi K3.1 Pro across NVIDIA GPUs including Vera Rubin, AMD Instinct MI355X, Google Cloud TPUs, and the Zhenwu 810e and Zhenwu M890. Launch images are available for NVIDIA CUDA 13.0 and 13.4 and for AMD ROCm. The model was built with Moonshot AI, Inferact and the vLLM community.
- vLLMOfficialvLLM details K3.1 serving with token-sharded prefill and fused decode
AIvLLM says K3.1 combines DSA sparse attention with recurrent Kimi Delta Attention, storing MLA latents, indexer keys and KDA state in one packed cache. Its serving work adds token-sharded prefill, exact top-k merging for decode context parallelism, and a fused low-latency decode path.
- vLLMOfficialvLLM adds day-zero serving for Kimi K3.1 Pro with 1M-token context
AIvLLM says Kimi K3.1 Pro, from Moonshot AI, is supported for serving on day zero. The post cites a 1M-token context window and a 65.7% score on Terminal-Bench 4.0 with the Claude Code harness. vLLM also lists day-zero serving across GPUs and TPUs plus RL support.
- vLLM BlogOfficialPickvLLM adds day-0 support for Kimi K3.1 Pro on NVIDIA, AMD, TPU and T-Head
AIvLLM says it supports Moonshot AI's Kimi K3.1 Pro on day 0 across NVIDIA GPUs, AMD GPUs, Google TPUs and Alibaba T-Head PPUs. The post says the model adds Sparse Attention (DSA) to all 24 full-attention layers, and that vLLM packs its MLA latent, indexer keys and KDA state into one KV cache. vLLM also reports up to 1.69× faster time to first token than head-sharded prefill at 340K tokens on 8× GB300.