On Tuesday, we brought together partners and friends in San Francisco to launch Artificial Analysis Intelligence Index v5 and our new Progress Index
Micah Hill-Smith (@_micah_h) introduced Intelligence Index v5, including two new benchmarks and three upgrades, with greater weighting on agentic coding and science. For the first time, the majority of the Index is based on private test sets or held-out answers.
We also introduced Artificial Analysis Progress Index, which uses item response theory to estimate a general capability score for each model across dozens of evaluations measured over time, including those retired or excluded from the Intelligence Index. It builds on prior work from @EpochAIResearch and @AnthropicAI to place models on a single capability scale. Intelligence Index remains the flagship metric for choosing which current model to use for a use case today; Progress Index is the better tool for plotting trends over time.
George Cameron (@grmcameron) led a discussion with Silas (@silasalberti) from @cognition, Steven (@StevenDillmann) from @Stanford and Terminal-Bench-Science, and our own Declan Jackson about building evaluations as models become more capable.
The evening also covered our work on 3D modeling, personality, cyber defense, custom enterprise evaluations and robotics. Thank you to our speakers, partners and everyone who joined us!
Explore Intelligence Index v5: https://artificialanalysis.ai/
Explore Progress Index: https://artificialanalysis.ai/trends