I'd frame it as follows: The data industry has taken off in recent years, but the quality of our net output is still far too low (amplifying behaviors like reward hacking).
We're working to build scalable methods for creating sample-efficient data for RL. In order to keep this pipeline going, we need to push the frontier of evaluation science, while building specific benchmarks to hillclimb on areas of clear economic value.
I've been advising Mercor on how to build this research direction effectively. These are my views, but I'm confident we're going to see a major investment from economy around the frontier labs (open inference, open post-training, and data) orient around expertise in building real-world representative evals and synthetic data methods to scaling training data around them.
I’m personally very excited about this, it is the research that will make more of the economy “feel the AGI” for the first time.
(And, there’s a big opportunity to build this on open models.)