Skip to content
Trending storyDeveloping

SGLang describes long-context serving optimizations for GLM-5.3-Flash on H200 GPUs

1 article1 sourcesince Nov 9Last article 6h ago ·

Overview

AISummary of 1 article

SGLang, the LLM serving framework from LMSYS Org, reports that Ant International AI Infra's long-context serving optimizations raise GLM-5.3-Flash throughput on H200 GPUs.

The LMSYS Org post on X gives the figures: on 4×H200, prefill throughput rises 2.06× for 64K-token inputs without a shared prefix, and decode throughput rises 4.32× for 64K/1K requests with 99.9% shared prefix. On 16×H200, end-to-end throughput rises 2.11–4.28× across three prefix-sharing ratios.

These are the vendor-reported benchmark results cited in the LMSYS post, not independently verified measurements.

Written by AI from the articles below · updated Nov 9, 4:12 PM ET

Check the sources:

Latest developmentNov 9, 2:42 PM ET
SGLang reports 2.11–4.28× GLM-5.3-Flash throughput gains on H200 GPUs

Article timeline

The articles in this story. Times are ET.

Nov 9
  1. LMSYS OrgOfficial
    SGLang reports 2.11–4.28× GLM-5.3-Flash throughput gains on H200 GPUs

    AIAnt International AI Infra reports that its long-context serving optimizations for SGLang raise GLM-5.3-Flash throughput on H200 GPUs. On 4×H200, prefill throughput rises 2.06× for 64K-token inputs without a shared prefix, and decode throughput rises 4.32× for 64K/1K requests with 99.9% shared prefix. On 16×H200, end-to-end throughput rises 2.11–4.28× across three prefix-sharing ratios.

    Image from @lmsysorg's post

Heat Trend

Current comparable heat
9
Peak comparable heat
9Nov 9, 5:00 PM ET
Change in past 24 hours
–

The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.