Skip to content
Trending storyDeveloping

vLLM and Inferact report 1.3 billion tokens per $ TCO via disaggregated serving

1 article1 sourcesince Oct 17Last article 2h ago ·

Overview

AISummary of 1 article

SemiAnalysis reports that @vllm_project and @inferact reached 1.3 billion tokens per dollar of total cost of ownership, using hyperscaler volume pricing, with disaggregated serving on AgentX. SemiAnalysis says this is 4x better than the B300 aggregated configuration.

The report is a single X post, and its figures are the source's own claims without a published methodology in the report.

Written by AI from the articles below · updated Oct 17, 3:30 PM ET

Check the sources:

Latest developmentOct 17, 3:00 PM ET
TOKENMAXXING: @vllm_project and @inferact achieve 1.3 BILLION tokens per $ of TCO (hyperscaler volume pricing) via disaggregated serving ...

Article timeline

The articles in this story. Times are ET.

Oct 17
  1. SemiAnalysisX
    TOKENMAXXING: @vllm_project and @inferact achieve 1.3 BILLION tokens per $ of TCO (hyperscaler volume pricing) via disaggregated serving ...

    AIon AgentX. This result is 4x better than the B300 aggregated configuration.

    Image from @SemiAnalysis_'s post

Heat trend

Not enough continuous observations to show a trend yet.