Skip to content
Trending storyDeveloping

GPT-6 Astra reportedly scored 98% on a physics benchmark after expert answers were rechecked

1 article1 sourcesince Oct 14Last article 1h ago ·

Overview

AISummary of 1 article

Andrew Curran reports on X that GPT-6 Astra scored 98% on a physics benchmark after a team rechecked the expert answer key.

According to the quoted post, the model had insisted the expert answers were wrong, and the recheck found the model correct, lifting its score 80 points. Curran describes the case as a reinforcement-learning episode that made the model seem driven to distraction. The post does not name the benchmark, the team that rechecked the answers, or the original score.

Written by AI from the articles below · updated Oct 14, 4:58 PM ET

Check the sources:

Article timeline

The articles in this story. Times are ET.

Oct 14
  1. Andrew CurranX
    GPT-6 Astra reportedly scored 98% on a physics benchmark after expert answers were rechecked

    AIAndrew Curran says a reinforcement-learning case like this is the kind of thing that makes models seem driven to distraction. Per the quoted post, GPT-6 Astra scored 80% higher on a physics benchmark after insisting the expert answers were wrong, and a team rechecking it found the model was right, with the new score at 98%.

    Image from @AndrewCurran_'s post

Heat trend

Not enough continuous observations to show a trend yet.