We’ve added DeepSeek V4.1 Flash to AA-AgentPerf, with results for NVIDIA and AMD hardware. At 100 tokens/s per user, NVIDIA GB300 is in first place with throughput up to 261M tokens/s per MW.
AA-AgentPerf is our agentic inference performance benchmark. It replays real agentic coding sessions, including prompt caching and tool-call delays, to give a realistic measure of inference performance in day-to-day agentic workloads. We measure total system throughput per megawatt of runtime power, and per-agent output speed at varying concurrency levels, to show the trade-off between system utilization and individual user experience.
DeepSeek V4.1 Flash joins the leaderboard today. It brings the leaderboard to four featured models, joining DeepSeek V4 Pro 0813, Kimi K3, and GLM-5.2/5.3. We are evolving the featured models as popular new open-weights models release, and will feature day-0 results for major future releases.
Background on DeepSeek v4.1 Flash:
➤ It scores 39 on the Artificial Analysis Intelligence Index, ahead of DeepSeek’s earlier V4 Pro 0813 at 36, with about a third of the parameters (552B ‘backbone’ and 16B active, against 1.6T and 49B)
➤ It is built on a new architecture, with a causal encoder-decoder design that activates 8B parameters per input token and 16B per output token. It uses a further 196B parameters of Engram n-gram memory that can sit in host memory rather than on the accelerator, and compressed sparse attention that needs about a quarter of even the earlier V4 Flash model’s KV cache per token
➤ This makes it much more efficient to serve, since the weights fit on fewer GPUs and the smaller KV cache leaves room for more concurrent agents per GPU
Initial AA-AgentPerf results for DeepSeek V4.1 Flash:
➤ V4.1 Flash runs on as few as 2 GPUs, so rack-scale systems get less advantage over 8-GPU single nodes like B300, which stays within about 20% of rack-scale GB300 NVL72 across the curve, and leads at both ends
➤ At 100 tok/s per user, GB300 leads with ~261M tokens/s per MW, ahead of B300 (~220M), B200 (~193M), GB200 (~178M) and MI355X (~128M)
➤ Every Blackwell system and MI355X can reach around 600 to 670 tok/s output speed for a single agent (ie. at batch size of one), against 372 to 392 tok/s on H100 and H200
➤ V4.1 Flash is both more intelligent than V4 Pro 0813 and much cheaper to serve, yielding ~1.3x (GB300) to ~6.2x (B200) the throughput per MW at 100 tok/s per user compared to its larger cousin