Epoch AI moves its Capabilities Index to Bayesian methods, ranks models probabilistically
Article timeline
The articles in this story. Times are ET.
Epoch AIOfficialEpoch AI updates its ECI results and methodology on its websiteAIEpoch AI says it has updated its ECI results and methodology, which are available on its website at The post gives no model names, scores, or figures.
- Epoch AIOfficialEpoch AI updates its benchmark methodology with a Bayesian update
AIEpoch AI says it has made several changes to its methodology as part of a Bayesian update, including benchmark-specific noise terms. The company links to a full report explaining the updates and what motivated them.
- Epoch AIOfficialEpoch AI finds reasoning models' capability trend steeper, 1-2 ECI/year
AIEpoch AI says frontier trends remain very similar, with a clear trend break between reasoning and non-reasoning models. Both trends are now slightly steeper, by 1-2 ECI per year.
- Epoch AIOfficialEpoch AI gives probabilistic capability rankings for Astra, Opus 5.5 and Fable 5.1
AIEpoch AI now produces uncertainty estimates and a joint distribution over all models' ECIs, enabling direct model comparisons. It says Astra has a 72% probability of being more capable than Opus 5.5 and a 99% probability of being more capable than Fable 5.1.
- Epoch AIOfficialEpoch AI updates its Capabilities Index to use Bayesian methodology
AIEpoch AI has updated its Epoch Capabilities Index (ECI) to use a Bayesian methodology, which improves uncertainty estimates and allows more accurate model comparisons even when individual ECI intervals overlap. Scores for frontier models are largely unchanged, with models stronger than GPT-4 moving less than 1 ECI point on average.
- Epoch AIOfficialEpoch AI updates its Capabilities Index to use a Bayesian model
AIEpoch AI says it has updated the Epoch Capabilities Index (ECI), which tracks AI model capability using benchmark results, to a Bayesian methodology. The change is meant to provide better-justified uncertainty estimates and to calculate how likely one model is to be more capable than another. The ECI is based on Item Response Theory, and the previous version used a frequentist fit with bootstrapped confidence intervals.