Skip to content
View original post on X: SGLangOfficial
Published
32/100

SGLang matches DeepInfer on prefill and decode throughput

AISGLang says it now matches DeepInfer performance on both prefill and decode. It reports up to 1.04× prefill throughput on 4× GB300 and 4× B300, and up to 7.4% decode throughput on 64× GB300.
SGLangVerified on XOfficial
@sgl_project
View on X

DeepInfer set the bar. SGLang pushed it higher! 🚀

SGLang now matches DeepInfer performance on BOTH prefill and decode:

  • Prefill: Up to 1.04× throughput on 4× GB300 and 4× B300
  • Decode: Up to 7.4% throughput on 64× GB300

We also provide an end-to-end reproduction guide and a full serving stack to run it in production.

Source details

SGLang · x.comPublished · added here

Original title: DeepInfer set the bar. SGLang pushed it higher! 🚀

Topics

Open-source ecosystemDeployment & engineeringProduct updates