AMD and Embedded LLM report MI355X matches B300 per-GPU throughput on Kimi K3
Overview
AMD and Embedded LLM say an eight-GPU AMD MI355X system serving Kimi K3 with vLLM delivers 24.4% more total tokens per second per GPU than an NVIDIA HGX B300 at 50 tokens/s/user P90 interactivity on the AgentX benchmark.
Under the benchmark's modeled ownership cost, they say the MI355X delivers 87.4% more tokens per dollar. The vLLM blog post attributes the gains to decode context parallelism, MoE output deduplication, kernel fusion, MLA decode padding and KDA parallelization.
The results come from the companies' own post on the vLLM blog, and the cost comparison depends on the benchmark's ownership cost model.
Written by AI from the articles below · updated Oct 15, 10:11 PM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
- vLLM BlogOfficialPickHow AMD and Embedded LLM scaled Kimi K3 on MI355X to match B300 throughput per GPU
AIAMD and Embedded LLM report that an eight-GPU MI355X system serving Kimi K3 with vLLM delivers 24.4% more total tokens per second per GPU than NVIDIA HGX B300 at 50 tokens/s/user P90 interactivity on the AgentX benchmark. Under the benchmark's modeled ownership cost, it delivers 87.4% more tokens per dollar. The post explains the optimizations behind these results, including decode context parallelism, MoE output deduplication, kernel fusion, MLA decode padding, and KDA parallelization.
Heat trend
Not enough continuous observations to show a trend yet.