# Prime Intellect releases Prime Mega MoE with overlapping backward kernel

> Original title: MoE layers usually route, send, compute, and return through separate steps, leaving GPUs waiting on communication.

- Source: Prime Intellect
- Published: 2026-10-29T17:41:35.000Z
- Bad Influence: https://badinfluence.news/items/ynnsw4wd0cekt7xpmu1w39b4w
- Original: https://x.com/PrimeIntellect/status/2115861617443569923

## Summary

Prime Intellect says Prime Mega MoE fuses MoE routing, dispatch, compute, and return into fast overlapping kernels that keep GPUs from waiting on communication. It extends DeepSeek's Mega MoE, which fuses these steps in the forward pass, by adding a backward kernel that covers both passes.

## Post

MoE layers usually route, send, compute, and return through separate steps, leaving GPUs waiting on communication.

DeepSeek's Mega MoE fuses these steps into one kernel, overlapping communication with compute in the forward pass.

Prime Mega MoE adds a fast, overlapping backward kernel, combining forward and backward passes.

![](https://badinfluence.news/api/img-proxy?u=https%3A%2F%2Fpbs.twimg.com%2Fmedia%2FHV0M7aMbkAAXxRi.png&mode=thumb&exp=1793491200&sig=3d1cccce9365b611)