# Dwarkesh Patel says data gains outpaced model gains in pretraining progress

> Original title: This got totally buried cause it came out the same day as Navier Stokes.

- Source: Dwarkesh Patel
- Published: 2026-11-11T03:33:42.000Z
- Bad Influence: https://badinfluence.news/items/d0144yomnozpodxoih6rc0ej1
- Original: https://x.com/dwarkesh_sp/status/2120359285398155623

## Summary

Dwarkesh Patel and Jerbear pretrained combinations of year-representative open model recipes and data corpora from 2019 to 2025 at small scales. Data improvements contributed 12.0x compute multipliers versus 3.7x for model improvements, or 3.24x as much. The author says the gains stack independently, with a better dataset helping every architecture about equally, and links the full results on dwarkesh.com.

## Post

This got totally buried cause it came out the same day as Navier Stokes.

But I think this is super interesting evidence about what the true major source of algorithmic progress is.

## Quoted post @dwarkesh_sp

> Pretraining progress seems to be coming mostly from data improvements.
> 
> @who_is_jerbear and I pretrained combinations of year-representative open model recipes and data corpuses across 2019 to 2025 at various small scales.
> 
> Data improvements contributed 3.24x as many compute multipliers as model improvements did (12.0x vs 3.7x).
> 
> And the gains stack independently - a better dataset helps every architecture about equally, and vice versa.
> 
> Here are full results, plus what we think this means for the future of AI progress:
> 
> https://www.dwarkesh.com/p/pretraining-progress-is-mostly-data

https://x.com/dwarkesh_sp/status/2097359573581799481