Prime Intellect extends DeepSeek's Mega MoE kernel with a fast backward pass for training
AIPrime Intellect says it extended DeepSeek's Mega MoE kernel with a fast, overlapping backward pass, making it usable for MoE training. The company reports up to 3.11× faster MoE kernels than the default PyTorch implementation and up to 1.34× faster end-to-end training versus the Prime-RL baseline.
AIWhy it matters.
The post explains how overlapping communication with compute in a fused kernel cuts MoE training overhead, with per-layer and end-to-end benchmark numbers for checking the gains.
Source details
Prime Intellect Blog · primeintellect.aiPublished · added here
Original title: Prime Mega MoE: Fast Fused MoE Kernels for Training