Skip to content
Trending storyDeveloping

Artificial Analysis: HLE-Diamond top models score 16-35 points higher on reasoning than knowledge

1 article1 sourcesince Oct 29Last article 1h ago ·

Overview

AISummary of 1 article

Artificial Analysis reports that on HLE-Diamond, every model scoring above 50% gets 16-35 points more on reasoning questions than on knowledge questions.

The benchmark labels each question as either reasoning or knowledge.

Among the leaders, GPT-6.1 Sol (max) shows the widest gap, scoring 76% on reasoning and 41% on knowledge, according to Artificial Analysis.

Written by AI from the articles below · updated Oct 29, 10:14 PM ET

Check the sources:

Latest developmentOct 29, 10:09 PM ET
HLE-Diamond shows top models score 16-35 points higher on reasoning than knowledge

Article timeline

The articles in this story. Times are ET.

Oct 29
  1. Artificial AnalysisOfficial
    HLE-Diamond shows top models score 16-35 points higher on reasoning than knowledge

    AIArtificial Analysis says HLE-Diamond labels each question as reasoning or knowledge. Every model scoring above 50% does 16-35 points better on reasoning questions than on knowledge questions. GPT-6.1 Sol (max) has the widest gap among the leaders, at 76% on reasoning and 41% on knowledge.

    Image from @ArtificialAnlys's post

Heat Trend

Not enough continuous observations to show a trend yet.