# Private benchmarks may overstate gaps between open and closed models

> Original title: Private benchmarks often show open models are further behind, but the problem with private benchmarks is you don’t have the community to ...

- Source: Nathan Lambert
- Published: 2026-10-28T13:35:59.000Z
- Bad Influence: https://badinfluence.news/items/qfprbhbkbt3909z3wuktk2ja4
- Original: https://x.com/natolambert/status/2115437422146130032

## Summary

Nathan Lambert says private benchmarks often show open models trailing further behind. He argues they lack the community that finds and fixes bugs, and says he is split 50/50 on whether private or open benchmarks are the better approach.

## Post

Private benchmarks often show open models are further behind, but the problem with private benchmarks is you don’t have the community to squash the inevitable bugs for you. I'm 50/50 on if I think private or open-source is the best approach.

![](https://badinfluence.news/api/img-proxy?u=https%3A%2F%2Fpbs.twimg.com%2Fmedia%2FHVuLIBkWEAAi4Tg.jpg&mode=thumb&exp=1793404800&sig=106a91208b13b93c)