Skip to content
Trending storyDeveloping

AI models outperform untrained reviewers at finding planted errors in trial papers

1 article1 sourcesince Oct 28Last article 1h ago ·

Overview

AISummary of 1 article

A preprint by @rust_ruslan tested 15 large language models against untrained human reviewers on planted methodological errors in clinical trial reports.

Untrained humans averaged 27% detection of major errors, according to the preprint. On the original reports the models scored 53% to 96%, and on 58 trials published in 2026 they scored 58% to 99%, with Claude Opus 5.5 highest at 99%.

The authors say humans caught text-table contradictions only 7% of the time. Review took 5 to 8.5 hours per human, compared with 1.4 to 16.3 minutes for the models.

Written by AI from the articles below · updated Oct 28, 8:22 PM ET

Check the sources:

Latest developmentOct 28, 8:18 PM ET
LLMs catch more planted errors in clinical trial papers than untrained human reviewers

Article timeline

The articles in this story. Times are ET.

Oct 28
  1. Rohan PaulX
    LLMs catch more planted errors in clinical trial papers than untrained human reviewers

    AIA preprint tested 15 large language models on planted methodological errors in clinical trial reports, with untrained human reviewers averaging 27% major-error detection. The models scored 53% to 96% on the original reports and 58% to 99% on 58 trials published in 2026, with Claude Opus 5.5 highest at 99%. The authors say humans caught text-table contradictions only 7% of the time, and review took 5 to 8.5 hours per human versus 1.4 to 16.3 minutes for the models.

    Image from @rohanpaul_ai's post

Heat Trend

Not enough continuous observations to show a trend yet.