Skip to content
Trending storyDeveloping

SGLang nears 1000 tok/s per user on GB300 NVL72 with DeepSeek Flash v4.1

1 article1 sourcesince Oct 29Last article 1h ago ·

Overview

AISummary of 1 article

SemiAnalysis says SGLang on GB300 NVL72 can get close to 1000 tokens per second per user running DeepSeek Flash v4.1, by aggressively fusing kernels together.

The post credits the SGLang, RadixArk, and NVIDIA teams with the work and says TileRT started the megakernel trend for open-source frontier models, which SGLang has widely adopted. The figure is SemiAnalysis's reported result, and the post does not give benchmark conditions.

Written by AI from the articles below · updated Oct 29, 11:21 AM ET

Check the sources:

Latest developmentOct 29, 11:01 AM ET
SGLang nears 1000 tok/s per user on GB300 NVL72 with DeepSeek Flash v4.1

Article timeline

The articles in this story. Times are ET.

Oct 29
  1. SemiAnalysisX
    SGLang nears 1000 tok/s per user on GB300 NVL72 with DeepSeek Flash v4.1

    AISemiAnalysis says SGLang on GB300 NVL72 can approach 1000 tokens per second per user running DeepSeek Flash v4.1, by aggressively fusing kernels. It credits TileRT with starting the megakernel trend for open-source frontier models and praises SGLang's adoption of it, along with work from the SGLang, RadixArk, and NVIDIA teams.

    Image from @SemiAnalysis_'s post

Heat Trend

Not enough continuous observations to show a trend yet.