Skip to content
View original post on X: Artificial AnalysisOfficial
Published
32/100

Artificial Analysis finds GPT-6 Astra more consistent than Kimi K3.1 Pro on VIALS

AIArtificial Analysis reports that Kimi K3.1 Pro solves 119 of 161 VIALS tasks at least once across three attempts, versus 118 for GPT-6 Astra. Requiring all three attempts to succeed, Kimi passes 73 tasks (45%) and Astra passes 101 (63%).
Artificial AnalysisVerified on XOfficial
@ArtificialAnlys
View on X

Part of a thread · earlier post

Solving a task once does not mean a model will answer it consistently, and consistency is a key part of the advantage for frontier models on VIALS

Kimi K3.1 Pro answers 119 of 161 tasks correctly at least once across three attempts, compared with 118 of 161 tasks for GPT-6 Astra. Requiring all three attempts to succeed changes the picture: Kimi passes 73 tasks (45%), while Astra passes 101 (63%)

Source details

Artificial Analysis · x.comPublished · added here

Original title: Solving a task once does not mean a model will answer it consistently, and consistency is a key part of the advantage for frontier models...

Topics

OpenAI / ChatGPTKimi / Moonshot AIChina AIEvals & benchmarks