Google DeepMind trains Gemma-4 on moral arguments; order flips fall, accuracy does not rise

1. 12:23 PM
   Rohan Paul@rohanpaul_aiX

   Training Gemma-4 to argue morally makes it steadier, not more accurate

   AIGoogle DeepMind trained Gemma-4 4B with reinforcement learning to argue moral dilemmas, using a self-judged pairwise reward. Answer flips under option reordering fell from 14.3% to 6.6% on ambiguous MoralChoice cases. Accuracy on moral fables dropped from 56.5% to 53.4%, and sycophancy barely changed, from 45.3% to 44.0%.

   

1. 3:59 PM
   elvis@omarsar0X

   DeepMind paper finds RL-trained moral reasoning gains order consistency, not robustness

   AIGoogle DeepMind trains a model with reinforcement learning on moral dilemmas from classic novels, using self-judged argument tournaments. The trained model becomes more consistent when answer options are reordered, but user pushback and reframing still change its judgments. The post's author argues that order invariance is easy to optimize and measure without improving underlying judgment, so self-rewarded or consistency metrics should be checked separately for robustness to pressure.

   

The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.