Recommended read.
Always great to have new ways to measure the impact of frontier intelligence. What strikes me the most is how much better Gemini (now with Carbon) has gotten.
Don't count out Google just yet.
As we move away from standard benchmarks, you will start to see all sorts of interesting results and surprises that tell a different story about these models' capabilities and how they could unlock economic value.
Carbon, in particular, is a very strong model for coordination and persistence, which is becoming increasingly clear in some of these new benchmarks.