Skip to content
View original post on X: SemiAnalysis
Published
38/100

NVIDIA AgentX submission claims 1000+ tok/s/user on 700B+ models

AINVIDIA's AgentX submission, according to SemiAnalysis, reports HBM chips reaching over 1,000 tokens per second per user on 700B+ parameter models using agent-swarm-written software. It also claims Rubin NVL72 delivers 20.9x better performance per TCO than GB300 NVL72 at 325 tokens per second per user, and up to 9.11x at 200 tokens per second per user.
SemiAnalysisVerified on X
@SemiAnalysis_
View on X

BREAKING NEWS🚨: HBM CHIPS CAN REACH OVER 1000 tok/s/user on giant 700B+ models via agent swarm-written software, according to a new NVIDIA AgentX submission.

Furthermore, on Rubin NVL72, it can achieve 20.9x better perf per TCO than GB300 NVL72 at even "ultrafast" speeds of 325 tok/s/user and up to 9.11x better perf per TCO at 200 tok/s/user. (1/2)🧵

Source details

SemiAnalysis · x.comPublished · added here

Original title: BREAKING NEWS🚨: HBM CHIPS CAN REACH OVER 1000 tok/s/user on giant 700B+ models via agent swarm-written software, according to a new NVID...

Topics

NVIDIAAI agentsAI codingDeployment & engineeringProduct updatesEvals & benchmarks