If you're training with TRL, you should really upgrade to v1.15. Our new Triton kernel scales SFT/RL to >100k tokens on the same GPU 🔥
TRL v1.15 adds Triton kernel scaling SFT and RL past 100k tokens
AISummary
Hugging Face's Lewis Tunstall says TRL v1.15 includes a new Triton kernel that scales SFT and RL training to over 100k tokens on the same GPU. He urges users of TRL to upgrade to the release.
Post on XView on X
Source: Lewis Tunstall @ COLM 🌉 · x.comPublished