# Epoch AI introduces an Inference Performance Index for comparing chip capacity

> Original title: Inference Performance Index: changing the way we measure inference capacity

- Source: Epoch AI
- Published: 2026-11-09T00:00:00.000Z
- Bad Influence: https://badinfluence.news/items/m91m61ujv1i1ml3rdty22an46
- Original: https://epoch.ai/publications/inference-performance-index

## Summary

Epoch AI introduces the Inference Performance Index (IPI), which scores chips by how many agentic requests they can serve at once across output speeds of 50 to 400 tokens per second per request. Epoch estimates GB300 offers about 25 times H100's inference capacity per chip on the modeled workloads, despite only 2.5 times the 8-bit FLOP/s, because memory and inter-chip communication matter more than arithmetic throughput.

## Why it matters

The report explains how memory and interconnect limits shape inference capacity, making chip comparisons based on raw FLOP/s less reliable.