Zyphra says cross-layer expert routing is predictable, enabling token shuffling with no extra network traffic
Overview
Zyphra reports that expert routing in mixture-of-experts models is predictable across layers, since the experts a token uses in one layer indicate which it will need next.
Its token shuffling method moves each token to the GPU holding those experts within a transfer that already runs after attention, adding no network traffic.
Written by AI from one article, by Zyphra
Check the sources:
Article timeline
Follow the coverage from different perspectives. Times are ET.
- ZyphraToken shuffling routes tokens to predicted experts without extra network traffic
AIZyphra reports that expert routing in mixture-of-experts models is predictable across layers, since the experts a token uses in one layer indicate which it will need next. Its token shuffling method moves each token to the GPU holding those experts within a transfer that already runs after attention, adding no network traffic.
Heat trend
Not enough continuous observations to show a trend yet.