Prime Intellect releases Prime Mega MoE, a trainable fused MoE kernel
AIPrime Intellect has released Prime Mega MoE, a training extension of DeepSeek's Mega MoE kernel that adds a fused, overlapping backward pass. Self-reported tests on Qwen3 MoE models show full training steps 32% to 34% faster than the prime-rl baseline and 48% to 55% faster than TorchTitan, with the forward pass up to 3.11x faster on a 30B layer.
Loading source posts…
Source details
MarkTechPost · Asif Razzaq · marktechpost.comPublished · added here
Original title: Prime Intellect Releases Prime Mega MoE: Trainable Fused MoE Kernel, 3.11x Faster Forward Pass