Skip to content
View original post on X: SGLangOfficial
Published
38/100

SGLang doubles DeepSeek V4.1 Flash low-latency and throughput speeds

AISGLang says its MegaKernel optimizations raise DeepSeek V4.1 Flash interactivity from 538 to 943 tokens per second per user. Its WideEP and system-wide optimizations raise total throughput from 348K to 654K tokens per second per GPU, counting cache, new input, and output tokens.
SGLangVerified on XOfficial
@sgl_project
View on X

We’ve been working to push both ends of the curve on DeepSeek V4.1 Flash 🚀

  • For low latency, our MegaKernel optimizations take interactivity from 538 to 943 tok/s/user.
  • For high throughput, WideEP and system-wide optimizations take total throughput (cache + new input + output) from 348K to 654K tok/s per GPU.

There’s still lots of room for improvement. We’ll share more updates soon.

SemiAnalysis@SemiAnalysis_
Congrats to the SGLang and RadixArk team for their amazing work on MegaKernel + PD, improving both high concurrency throughput and low-latency serving as well! (1/2)🧵
View quoted post on X
Source details

SGLang · x.comPublished

Original title: We’ve been working to push both ends of the curve on DeepSeek V4.1 Flash 🚀

Topics

Open-source ecosystemDeployment & engineeringChina AIProduct updates