HLE-Diamond labels every question as reasoning or knowledge, and the models above 50% all score 16-35 points higher on reasoning tasks than knowledge. GPT-6.1 Sol (max) has the widest gap among the leaders, scoring 76% on reasoning and 41% on knowledge.
HLE-Diamond shows top models score 16-35 points higher on reasoning than knowledge
AIArtificial Analysis says HLE-Diamond labels each question as reasoning or knowledge. Every model scoring above 50% does 16-35 points better on reasoning questions than on knowledge questions.
Source details
Artificial Analysis · x.comPublished
Original title: HLE-Diamond labels every question as reasoning or knowledge, and the models above 50% all score 16-35 points higher on reasoning tasks th...