Introducing Prime Mega MoE.
We made DeepSeek's Mega MoE kernel trainable. The results on two Qwen3 models:
- Up to 55% faster MoE training than TorchTitan
- Up to 34% faster hosted training with prime-rl
Prime Intellect makes DeepSeek's Mega MoE kernel trainable
AIPrime Intellect says it made DeepSeek's Mega MoE kernel trainable. On two Qwen3 models, it reports up to 55% faster MoE training than TorchTitan and up to 34% faster hosted training with prime-rl.
@PrimeIntellect
Source details
Prime Intellect · x.comPublished · added here
Original title: Introducing Prime Mega MoE.