# Salesforce paper finds RL-trained domain experts cut distillation compute by 42%

> Original title: New Salesforce paper finds that distilling from a big external teacher usually costs more compute than plain RL, while merging RL-trained...

- Source: Rohan Paul
- Published: 2026-11-08T00:17:53.000Z
- Bad Influence: https://badinfluence.news/items/srbqgijs270xfzao9x2cm2neu
- Original: https://x.com/rohanpaul_ai/status/2119222842332271059

## Summary

A Salesforce paper compares on-policy distillation with GRPO by held-out accuracy per training FLOP. With an external post-trained teacher, GRPO generally won and distillation often inflated response length. Distilling per-domain GRPO experts into one student matched joint GRPO on math plus code with 42.0% fewer total FLOPs.

## Post

New Salesforce paper finds that distilling from a big external teacher usually costs more compute than plain RL, while merging RL-trained domain experts saves 42%.

On-policy distillation lets a student learn from a teacher's per-token feedback on its own outputs, and many expect this dense signal to beat RL. But every training token also needs a teacher forward pass.

They compared distillation and GRPO by held-out accuracy per training FLOP, varying task difficulty, model sizes, domains, and training settings.

With an external post-trained teacher, GRPO generally won, and distillation often inflated response length. Distilling per-domain GRPO experts into 1 student matched joint GRPO on math plus code with 42.0% fewer total FLOPs.

To save compute, train per-domain experts with RL and distill them into 1 model, rather than distilling from a big external teacher.

![](https://badinfluence.news/api/img-proxy?u=https%3A%2F%2Fpbs.twimg.com%2Fmedia%2FHWjzrtjbQAAfBld.jpg&mode=thumb&exp=1794355200&sig=4ce521b79f8450d0)