Skip to content
View original post on X: LMSYS OrgOfficial
Published
36/100

SGLang reports 2.11–4.28× GLM-5.3-Flash throughput gains on H200 GPUs

AIAnt International AI Infra reports that its long-context serving optimizations for SGLang raise GLM-5.3-Flash throughput on H200 GPUs. On 4×H200, prefill throughput rises 2.06× for 64K-token inputs without a shared prefix, and decode throughput rises 4.32× for 64K/1K requests with 99.9% shared prefix.
LMSYS OrgVerified on XOfficial
@lmsysorg
View on X

New blog: 2.11–4.28× Higher GLM-5.3-Flash Serving Throughput on H200

Ant International AI Infra brings faster long-context serving to SGLang. How much do cache reuse, speculative decoding, and kernel tuning add up to?
Engineering Result:
- 2.06× prefill throughput on 4×H200: 64K/10 tokens, no shared prefix.
- 4.32× decode throughput on 4×H200: 64K/1K tokens, 99.9% shared prefix.
- 2.11–4.28× end-to-end throughput on 16×H200: 64K/1K tokens across three prefix-sharing ratios.

Thanks to Ant International AI Infra and the SGLang community.

Read the full blog below

Source details

LMSYS Org · x.comPublished · added here

Original title: New blog: 2.11–4.28× Higher GLM-5.3-Flash Serving Throughput on H200

Topics

Open-source ecosystemDeployment & engineeringChina AITutorials & how-to