LMSYS launches SGLang Low-Latency Mode, raising DeepSeek-V4.1-Flash token speed on NVIDIA and AMD

AISGLang announced a Low-Latency Mode built with NVIDIA and AMD that raises P90 per-user token speed on DeepSeek-V4.1-Flash. In SemiAnalysis's AgentX trace replay, speed rose from 671 to 928 tok/s on NVIDIA GB300 (1.38x) and from 332 to 516 tok/s on AMD MI355X (1.56x). SemiAnalysis said merged megakernel support brings GB300 close to 1000 tok/s per user, with low-latency kernels for GLM-5.3, K3, and MiniMax M3 in progress.

AILMSYS Org announces SGLang Low-Latency Mode, built with NVIDIA and AMD. On DeepSeek-V4.1-Flash with DSpark, it raises P90 per-user token speed up to 1.38× on NVIDIA GB300 (671 to 928 tok/s) and 1.56× on AMD MI355X (332 to 516 tok/s) in SemiAnalysis's AgentX trace replay. The post also says low-latency kernels for GLM-5.3, K3, and MiniMax M3 are in progress and will plug in next.