Skip to content
View original post on X: SemiAnalysis
Published
32/100

NVIDIA and Inferact submit initial vLLM Rubin NVL72 benchmark result

AINVIDIA and Inferact have submitted an initial vLLM benchmark on Rubin NVL72 that the post calls state-of-the-art, with 6.5x better performance per TCO than GB300 NVL72. The submission uses a new locality-aware MoE feature in CUDA 13.4, which the post says reduces latency and increases realized HBM bandwidth by about 1.2x.
SemiAnalysisVerified on X
@SemiAnalysis_
View on X

BREAKING NEWS: Less than 3 weeks since the MiniMax M3.1 Flash launch, NVIDIA and @Inferact have already submitted an initial vLLM Rubin NVL72 submission that is state-of-the-art and delivers 6.5x better perf per TCO than GB300 NVL72.

Great to see that Rubin is already becoming more and more mature and integrated in partnership with @vllm_project!

This Rubin submission uses the new locality-aware MoE feature in CUDA 13.4, which reduces latency and increases realized HBM bandwidth by ~1.2x.

Source details

SemiAnalysis · x.comPublished · added here

Original title: BREAKING NEWS: Less than 3 weeks since the MiniMax M3.1 Flash launch, NVIDIA and @Inferact have already submitted an initial vLLM Rubin N...

Topics

NVIDIADeployment & engineeringChina AIEvals & benchmarks