Skip to content
Trending storyDeveloping

SGLang reports 540+ tokens/s per user on 8×B300 and Geometry3K accuracy gain

1 article1 sourcesince Oct 15Last article 2h ago ·

Overview

AISummary of 1 article

SGLang (@sgl_project) says it reaches over 540 tokens/s per user at batch size 1 on 8×B300 for agentic workloads.

It also says it sustains over 60 tokens/s per user with 128 concurrent requests.

Separately, SGLang says that using Miles, it raised Geometry3K accuracy from 88.2% to 94.9% over 30 training updates. These are the company's own reported results, posted on X, and no independent benchmark is cited.

Written by AI from the articles below · updated Oct 15, 1:53 PM ET

Check the sources:

Latest developmentOct 15, 1:37 PM ET
SGLang hits 540+ tokens/s per user on 8×B300 at batch size 1

Article timeline

The articles in this story. Times are ET.

Oct 15
  1. SGLangOfficial
    SGLang hits 540+ tokens/s per user on 8×B300 at batch size 1

    AISGLang delivers over 540 tokens/s per user at batch size 1 on 8×B300 for agentic workloads, and sustains over 60 tokens/s per user with 128 concurrent requests. Using Miles, SGLang raises Geometry3K accuracy from 88.2% to 94.9% in 30 training updates.

Heat trend

Not enough continuous observations to show a trend yet.