# Prime Intellect extends DeepSeek's Mega MoE kernel with a fast backward pass for training

> Original title: Prime Mega MoE: Fast Fused MoE Kernels for Training

- Source: Prime Intellect Blog
- Published: 2026-10-23T04:00:00.000Z
- Bad Influence: https://badinfluence.news/items/rdh9a1658dy137i0uoapcc8tq
- Original: https://www.primeintellect.ai/blog/prime-mega-moe

## Summary

Prime Intellect says it extended DeepSeek's Mega MoE kernel with a fast, overlapping backward pass, making it usable for MoE training. The company reports up to 3.11× faster MoE kernels than the default PyTorch implementation and up to 1.34× faster end-to-end training versus the Prime-RL baseline. Supported data types are bfloat16, MXFP8 on Blackwell, and blockwise FP8 on Hopper.

## Why it matters

The post explains how overlapping communication with compute in a fused kernel cuts MoE training overhead, with per-layer and end-to-end benchmark numbers for checking the gains.