Artificial Analysis finally moves to closed benchmarks (goodbye GDPval-AA), and finds that open weights models are falling further behind than they had previously measured. Take with a grain of salt, but matches the general vibes that we haven't seen a Fable-class open model.
Artificial Analysis finds open-weights models falling further behind closed models
AIArtificial Analysis moves to closed benchmarks and drops GDPval-AA, finding open-weights models trail the frontier by more than earlier measurements showed. Its v5 index puts Kimi K3.1 Pro, whose weights are not yet released, at 46, 12 points ahead of the best released-weights models, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max), which score 32.
Ethan MollickVerified on X
@emollick
Open weights models fall further behind in v5. The leader, Kimi K3.1 Pro (weights not yet released), scores 46, 12 points behind the leader, compared with a 3-point gap in v4.3. Of models with released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) score highest at 32, and Claude Haiku 5.5 (max) outscores all of them. The gap is widest on HLE-Diamond and Terminal-Bench-Science, where Kimi K3.1 Pro scores 37% and 34%, compared with 68% for Claude Opus 5.5 (max) and 63% for GPT-6 Astra (max).
Source details
Ethan Mollick · x.comPublished
Original title: Artificial Analysis finally moves to closed benchmarks (goodbye GDPval-AA), and finds that open weights models are falling further behin...