# Max Planck study finds reasoning makes LLMs agree more, including on errors

> Original title: New Max Planck paper finds that turning on reasoning makes different LLMs agree more, even on mistakes, while long traces mark where they...

- Source: Rohan Paul
- Published: 2026-11-05T10:25:34.000Z
- Bad Influence: https://badinfluence.news/items/bl993q31vw0zgqcbkrv8u0ldd
- Original: https://x.com/rohanpaul_ai/status/2118288607106764857

## Summary

Max Planck researchers report that turning on reasoning makes 9 Qwen and GPT models agree more across 3 US census prediction tasks. The share of people all 9 models got wrong rose by 2.0 to 2.4 points. For people with the longest reasoning traces, adjusted agreement fell to about 0.31 to 0.39.

## Post

New Max Planck paper finds that turning on reasoning makes different LLMs agree more, even on mistakes, while long traces mark where they disagree.

When several LLMs judge the same person, a wrong answer from one model could be fixed by switching to another. That only works if the models do not all make the same mistakes.

The authors tested 9 Qwen and GPT models on 3 prediction tasks built from US census surveys, with reasoning on and off. They measured agreement beyond what each model's accuracy alone would predict.

With reasoning on, models agreed more, and the share of people all 9 models got wrong rose by 2.0 to 2.4 points. For people who drew the longest traces, adjusted agreement fell to about 0.31 to 0.39.

For decisions about people, check whether anyone gets a correct answer from your model set, since more agreement can also mean more shared errors.

![](https://badinfluence.news/api/img-proxy?u=https%3A%2F%2Fpbs.twimg.com%2Fmedia%2FHWWsONHagAA1lVW.png&mode=thumb&exp=1794096000&sig=30c0a8b5c32cffdd)