This is the exact type of thing Elon was talking about a few days ago that makes the models behave like are being driven insane when they run into situations like this during reinforcement learning.
GPT-6 Astra reportedly scored 98% on a physics benchmark after expert answers were rechecked
AISummary
Andrew Curran says a reinforcement-learning case like this is the kind of thing that makes models seem driven to distraction. Per the quoted post, GPT-6 Astra scored 80% higher on a physics benchmark after insisting the expert answers were wrong, and a team rechecking it found the model was right, with the new score at 98%.
Post on XView on X
Andrew CurranVerified on X
@AndrewCurran_
GPT-6 Astra got +80% on a Physics benchmark. It insisted the expert results were wrong and its calculations were right. A team checked. And yes. GPT-6 Astra was right. New score: 98%View quoted post on X
Source: Andrew Curran · x.comPublished · added here