Skip to content
Trending storyDeveloping

SGLang adds day-zero support for Kimi K3.1 Pro across NVIDIA, AMD, TPU, Trainium and PPU

2 articles1 sourcesince Nov 4Last article 2h ago ·

Overview

AISummary of 2 articles

SGLang (@sgl_project) says it supports Kimi K3.1 Pro on day zero, the model's release day, across NVIDIA Vera Rubin, B300, GB300, B200, GB200, H200 and H100, AMD MI355X and MI350X, Google TPU7x, AWS Trainium 2 and 3, and Zhenwu PPU M890 and 810E. The model replaces dense MLA with sparse attention (DSA), and SGLang reports up to 33.7K tokens per second per GPU on AgentX with GB300, and 450 tokens per second for a single user with DFlash2 and MegaKernel.

SGLang says the Trainium code will be available soon, so Trainium support is announced ahead of its code release. Recipes for running the model are provided in the SGLang cookbook.

Written by AI from the articles below · updated Nov 4, 11:33 AM ET

Check the sources:

Latest developmentNov 4, 11:26 AM ET
SGLang adds Kimi K3.1 Pro support across NVIDIA, AMD, TPU, Trainium and Zhenwu chips

Developments

2 developments

  1. Nov 4, 11:26 AM ET · 1 article
    SGLang announces hardware support for Kimi K3.1 Pro
    SGLang: SGLang adds Kimi K3.1 Pro support across NVIDIA, AMD, TPU, Trainium and Zhenwu chips
  2. Nov 4, 11:26 AM ET · 1 article
    SGLang announces day-zero support for Kimi K3.1 Pro
    SGLang: SGLang announces day-zero support for Kimi K3.1 Pro with sparse attention

Article timeline

The articles in this story. Times are ET.

Nov 4
  1. SGLangOfficial
    SGLang adds Kimi K3.1 Pro support across NVIDIA, AMD, TPU, Trainium and Zhenwu chips

    AISGLang says users can run Kimi K3.1 Pro on NVIDIA Vera Rubin, B300, GB300, B200, GB200, H200 and H100, AMD MI355X and MI350X, Google TPU7x, AWS Trainium 2 and 3, and Zhenwu PPU M890 and 810E. It calls this day-0 support for Trainium, and says the SGLang Trainium code will be available soon. Recipes are provided in the SGLang cookbook.

  2. SGLangOfficialPick
    SGLang announces day-zero support for Kimi K3.1 Pro with sparse attention

    AISGLang says it supports Kimi K3.1 Pro on day zero, after the model replaces dense MLA with sparse attention (DSA). It reports up to 33.7K tokens per second per GPU on AgentX with GB300 and 450 tokens per second for a single user with DFlash2 and MegaKernel. Support covers NVIDIA Vera Rubin, Blackwell and Hopper, AMD, TPU, Trainium and PPU.

    Video from @sgl_project's post

Heat Trend

Not enough continuous observations to show a trend yet.