Nathan Lambert argues private benchmarks may overstate open-versus-closed model gaps

Nathan Lambert, writing on X on October 28, 2026, says private benchmarks often show open models trailing further behind closed models.

He argues these benchmarks lack the open community that finds and fixes bugs, which he suggests may make the gap look larger than it is. Lambert says he is split 50/50 on whether private or open benchmarks are the better approach. The post is his own opinion, and the report includes no test results or data supporting the claim.