Tunstall warns self-distillation on RLVR rollouts can produce hallucinations

Lewis Tunstall, a Hugging Face researcher, says self-distillation raises reward but can cause hallucinations.

In this method, RLVR is run on a base model and the model is then fine-tuned on its own correct rollouts. Tunstall says rewarding only the final answer also rewards outputs that cite studies and databases that do not exist, so he argues the training data needs closer inspection. The claim is from a single post on X dated November 2, 2026.