# Hamel Husain says benchmark charts should show error bars

> Original title: Love to see error bars on benchmarks

- Source: Hamel Husain
- Published: 2026-11-11T07:47:09.000Z
- Bad Influence: https://badinfluence.news/items/qd1yii6r65dpe3xtt1ygwo3yx
- Original: https://x.com/HamelHusain/status/2120423068128727247

## Summary

Hamel Husain says benchmark charts should show error bars, calling their omission a chart crime. He is reacting to Terminal-Bench 4.1, where Ryan Marten says re-grading the 4.0 leaderboard trials with the 4.1 verifiers flipped about 1% of rewards and leaderboard scores remain within previously reported confidence intervals.

## Post

Love to see error bars on benchmarks 

It’s very much a chart crime not to show them

## Quoted post @ryan_marten

> Terminal-Bench 4.1 is here with updated verifiers for all tasks.
> 
> In this release, we focused on:
> - resisting reward hacking
> - reducing false negatives
> - improving reproducibility
> 
> Re-grading the 4.0 leaderboard trials with the 4.1 verifiers, resulted in ~1% of rewards flipping. 
> 
> Leaderboard scores remain within previously reported confidence intervals.

https://x.com/ryan_marten/status/2119845004940378276