Skip to content
View original post on X: InferactOfficial
Published
38/100

Inferact's vLLM on TPU v7 hits 391 tokens/s per user for Kimi K3

AIInferact says its Kimi K3 result on vLLM with TPU v7 reaches 391 tokens/s per user on SemiAnalysis AgentX. The result uses its TPU megakernels in vLLM TorchTPU on long-context agentic workloads, and the megakernels are open source. The results are publicly viewable on InferenceX.
InferactVerified on XOfficial
@inferact
View on X

Our Kimi K3 result running @vllm_project TPU v7 reaches 391 tokens/s per user on @SemiAnalysis_ AgentX.

This result uses our TPU megakernels in vLLM TorchTPU and holds up on real, long-context, production-level agentic workloads. The results are publicly viewable on InferenceX, and the megakernels are open source.

1/2

Source details

Inferact · x.comPublished · added here

Original title: Our Kimi K3 result running @vllm_project TPU v7 reaches 391 tokens/s per user on @SemiAnalysis_ AgentX.

Topics

Kimi / Moonshot AIOpen-source ecosystemDeployment & engineeringChina AIProduct updatesEvals & benchmarks