Skip to content
Trending storyDeveloping

Study finds LLM moral rankings flip when answer order is reversed

1 article1 sourcesince Nov 10Last article

Overview

AISummary of 1 article

Researchers tested five LLMs, including GPT-5.2, Gemini and DeepSeek, by asking which of two moral violations was worse.

Reversing the order of the two options changed the ranking in 26 of 30 test blocks, according to a report on X by Rohan Paul.

After averaging the results, four of the five models produced a clean, nearly identical ranking. That averaged ranking matched none of the individual test sets, the report says.

Written by AI from the articles below · updated Nov 10, 11:35 PM ET

Check the sources:

Latest developmentNov 10, 11:08 PM ET
LLM moral rankings often lack consistency and averaging can create false ones

Article timeline

The articles in this story. Times are ET.

Nov 10
  1. Rohan PaulX
    LLM moral rankings often lack consistency and averaging can create false ones

    AIResearchers tested five LLMs, including GPT-5.2, Gemini, and DeepSeek, on pairs of moral violations and asked which was worse. Reversing the order of the two options changed the ranking in 26 of 30 test blocks. After averaging, four of five models showed a clean, nearly identical ranking that matched no individual test set.

    Image from @rohanpaul_ai's post

Heat Trend

Current comparable heat
8
Peak comparable heat
10Nov 11, 12:00 AM ET
Change in past 24 hours
–

The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.