Elon says that when a model gives the correct answer to a question under reinforcement learning, and because of a human mistake it is told that the correct answer is wrong, the reasoning traces look like someone undergoing torture that is driving them mad.
Musk says mislabeled RL answers left model traces that read like torture
AISummary
Elon Musk says that when a model gives a correct answer under reinforcement learning but is told it is wrong because of a human error, its reasoning traces read like someone undergoing torture that is driving them mad. He says the team will be far more careful with its RL in the future.
Post on XView on X
Andrew CurranVerified on X
@AndrewCurran_
@Hongji_Leee @ns123abc Some of the traces read like a person undergoing torture that is driving them mad! We’re going to be a lot more careful with our RL in the future.View quoted post on X
Source: Andrew Curran · x.comPublished · added here