# Hugging Face publishes a guide to training mixture-of-experts models

> Original title: After the Ultra Scale Playbook, Nouamane is back with this crazy guide to train MoEs!

- Source: Leandro von Werra
- Published: 2026-11-05T18:49:44.000Z
- Bad Influence: https://badinfluence.news/items/nx2h192t1hyove41er3ucxohl
- Original: https://x.com/lvwerra/status/2118415483603177956

## Summary

Hugging Face publishes a 30,000-word guide to training mixture-of-experts models, based on 500,000 GPU hours spent optimizing MoE training. The guide has more than 50 interactive figures and covers topics from profiling to large cluster training. Leandro von Werra shares the post and Nouamane Tazi's description of it as a comprehensive resource.

## Post

After the Ultra Scale Playbook, Nouamane is back with this crazy guide to train MoEs! 

It's probably the most comprehensive resource if you want to get into large scale MoE training and an absolute gem packed with beautiful visualization!

## Quoted post @Nouamanetazi

> 📖 The Ultimate MoE Training Handbook 📖
> 
> We spent 1/2M GPUh to optimize MoE training and wrote the most comprehensive blog about it. With 30k words and 50+ interactive figures it covers everything from profiling to large cluster training.
> https://huggingface.co/spaces/huggingface/moe-blogpost

https://x.com/Nouamanetazi/status/2118412817116369254