Skip to content
Trending storyDeveloping

Inferact reports Kimi K3 at 391 tokens/s per user on vLLM TPU v7

1 article1 sourcesince Oct 20Last article 1h ago ·

Overview

AISummary of 1 article

Inferact says Kimi K3 running on vLLM with TPU v7 reaches 391 tokens/s per user on SemiAnalysis AgentX. The result uses Inferact's TPU megakernels in vLLM TorchTPU on long-context agentic workloads.

Inferact says the results are publicly viewable on InferenceX and that the megakernels are open source.

Written by AI from the articles below · updated Oct 20, 3:20 PM ET

Check the sources:

Article timeline

The articles in this story. Times are ET.

Oct 20
  1. InferactOfficial
    Inferact's vLLM on TPU v7 hits 391 tokens/s per user for Kimi K3

    AIInferact says its Kimi K3 result on vLLM with TPU v7 reaches 391 tokens/s per user on SemiAnalysis AgentX. The result uses its TPU megakernels in vLLM TorchTPU on long-context agentic workloads, and the megakernels are open source. The results are publicly viewable on InferenceX.

    Video from @inferact's post

Heat Trend

Not enough continuous observations to show a trend yet.