# Jason Wei charts which benchmarks frontier labs reported in 2026

> Original title: Created a "benchmark benchmark": a frequency distribution of which benchmarks were reported by the frontier labs in 2026: https://claude....

- Source: Jason Wei
- Published: 2026-11-10T22:31:55.000Z
- Bad Influence: https://badinfluence.news/items/jac0uql2qfu150c4fffb9frd4
- Original: https://x.com/_jasonwei/status/2120283338640621826

## Summary

Jason Wei created a frequency distribution of the benchmarks that frontier labs reported in 2026. He says HLE, SWE-Bench Pro, DeepSWE, and Terminal-Bench saw universal adoption, and that OpenAI produced the most benchmarks while Anthropic's only entry, BioMysteryBench, is rarely used.

## Post

Created a "benchmark benchmark": a frequency distribution of which benchmarks were reported by the frontier labs in 2026: https://claude.ai/artifact/22cBPJt6Aos6bYpjmB1BHw#distribution

Some observations:
- HLE, SWE-Bench Pro, DeepSWE, and Terminal-Bench saw universal adoption
- Scale AI is behind two of those four (SWE-Bench Pro, HLE w/ CAIS), plus MCP-Atlas (#9 benchmark)
- Of the frontier labs, OpenAI is the most prolific benchmark producer (SWE-bench Verified, BrowseComp, GDPval); people don't seem to mind evaluating on a benchmark made by a competitor
- Anthropic has just one on the chart (BioMysteryBench) that basically nobody uses 🤔
- In academia, strong showings from Princeton (SWE-bench, CharXiv), Stanford (Terminal-Bench), Berkeley (CyberGym, Agents' Last Exam), and NYU (GPQA)
- DeepSWE saw incredible adoption, from first appearing in June to all 12 labs reporting it
- Labs cycled through four versions of Terminal-Bench (2.0 → 4.0) in a single year

![](https://badinfluence.news/api/img-proxy?u=https%3A%2F%2Fpbs.twimg.com%2Fmedia%2FHWy-owZaoAEwC_0.jpg&mode=thumb&exp=1794700800&sig=a7bac52976d7a4a5)