Skip to content
Trending storyDeveloping

Tsinghua, Oxford and Stanford paper finds LLMs keep reasoning when told not to

1 article1 sourcesince Oct 10Last article 2h ago ·

Overview

AISummary of 1 article

A Tsinghua, Oxford and Stanford paper finds that LLMs still write out reasoning with thinking disabled, especially on open-ended questions.

With thinking off, DeepSeek-V4-Flash wrote out reasoning in 99.9% of open-ended answers.

Forcing answer-only replies on open-ended tasks raised compliance to about 40% across five models, according to the paper, but cut accuracy by about 15 points.

Written by AI from the articles below · updated Oct 10, 12:40 AM ET

Check the sources:

Latest developmentOct 10, 12:10 AM ET
LLMs keep reasoning out loud even with thinking disabled

Article timeline

The articles in this story. Times are ET.

Oct 10
  1. Rohan PaulX
    LLMs keep reasoning out loud even with thinking disabled

    AIA Tsinghua, Oxford, and Stanford paper finds that LLMs still write out reasoning with thinking turned off, especially on open-ended questions. With thinking disabled, DeepSeek-V4-Flash wrote reasoning in 99.9% of open-ended answers. Forcing answer-only replies on open-ended tasks raised compliance to about 40% across 5 models but cut accuracy by about 15 points.

    Image from @rohanpaul_ai's post

Heat trend

Not enough continuous observations to show a trend yet.