# CMU paper unifies LLM training stages under one likelihood-based rule

> Original title: The separate stages of AI training can collapse into a single method.

- Source: Rohan Paul
- Published: 2026-10-30T21:52:30.000Z
- Bad Influence: https://badinfluence.news/items/xwlx5df0tgbkge1ise6nu35ty
- Original: https://x.com/rohanpaul_ai/status/2116287151658901586

## Summary

Carnegie Mellon researchers recast MaxRL, a reinforcement learning method, as likelihood training on the model's own correct answers. This lets one trainer mix pretraining text, curated data, and RL samples in any proportion over time. On a 0.76B model, blending 5% midtraining data into math RL reached 32.69% on MBPP+, versus 29.81% for staged training.

## Post

The separate stages of AI training can collapse into a single method.

New Carnegie Mellon paper shows that pretraining, fine-tuning, and RL can share a single training rule, and blending their data often beats separate stages.

LLMs are usually trained in stages: pretraining, midtraining, fine-tuning, then RL, each with its own algorithm. Hard switches between stages can also erase earlier skills.

The authors recast a form of RL called MaxRL as likelihood training on the model's own correct answers. That lets one trainer mix pretraining text, curated data, and RL samples in any proportion over time.

On reasoning puzzles with a 124M model, the best blended runs beat the best staged run and kept general skills better. After later coding RL, a 0.76B model that blended in 5% midtraining data during math RL reached 32.69% on MBPP+, versus 29.81% for staged training.

Treat your training pipeline as a data-mixing schedule, and try blending earlier data into RL instead of switching stages abruptly.

![](https://badinfluence.news/api/img-proxy?u=https%3A%2F%2Fpbs.twimg.com%2Fmedia%2FHV6PiabbUAAAcXh.png&mode=thumb&exp=1793664000&sig=592626ea16a00817)