Skip to content
View original post on X: SGLangOfficial· 40/100AI score40/100

SGLang hits 540+ tokens/s per user on 8×B300 at batch size 1

AISummary

SGLang delivers over 540 tokens/s per user at batch size 1 on 8×B300 for agentic workloads, and sustains over 60 tokens/s per user with 128 concurrent requests. Using Miles, SGLang raises Geometry3K accuracy from 88.2% to 94.9% in 30 training updates.

Post on XView on X
SGLangVerified on X
@sgl_project

Part of a thread · earlier post

For agentic workloads on 8×B300, SGLang delivers over 540 tokens/s per user at batch size 1, while sustaining over 60 tokens/s per user with 128 concurrent requests.

With Miles, we improved Geometry3K accuracy from 88.2% to 94.9% in just 30 training updates.

Read our technical blog for details 👇
https://www.lmsys.org/blog/2026-10-15-meta-muse-spark-open

Source: SGLang · x.comPublished · added here

Topics

Meta / LlamaOpen-source ecosystemDeployment & engineeringEvals & benchmarksTutorials & how-to