Skip to content
View original post on X: SemiAnalysis
Published
22/100

SGLang nears 1000 tok/s per user on GB300 NVL72 with DeepSeek Flash v4.1

AISemiAnalysis says SGLang on GB300 NVL72 can approach 1000 tokens per second per user running DeepSeek Flash v4.1, by aggressively fusing kernels. It credits TileRT with starting the megakernel trend for open-source frontier models and praises SGLang's adoption of it, along with work from the SGLang, RadixArk, and NVIDIA teams.
SemiAnalysisVerified on X
@SemiAnalysis_
View on X

ALERT🚨: GB300 NVL72 @sgl_project can get close to 1000 tok/s/user on DeepSeek Flash v4.1 by aggressively fusing kernels together!!

TileRT started this trend in the ecosystem of megakernel engines on OSS frontier models! Glad to see SGLang widely adopting it!

Great work by the SGLang, @radixark, and NVIDIA teams on pushing performance!

Source details

SemiAnalysis · x.comPublished · added here

Original title: ALERT🚨: GB300 NVL72 @sgl_project can get close to 1000 tok/s/user on DeepSeek Flash v4.1 by aggressively fusing kernels together!!

Topics

DeepSeekAI codingOpen-source ecosystemDeployment & engineeringChina AIProduct updates