EUROPA consortium raises 1T-parameter MoE training throughput on JUPITER to about 1,300 tokens/s per GPU
Overview
The EUROPA consortium says it raised training throughput for a 1-trillion-parameter mixture-of-experts model on the JUPITER supercomputer from 90 to about 1,300 tokens/s per GPU, across 384 GH200 GPUs.
It describes the changes in a PyTorch blog post, including FlexShard placement for the Muon optimizer, offloading activation checkpoints to the Grace CPU, and running FSDP's all-gather on the compute stream.
The post is first-party, so the throughput figures are the consortium's own measurements. JUPITER is the EuroHPC exascale system hosted at Forschungszentrum Jülich, and the consortium presents the result as evidence that public infrastructure can support frontier-scale pretraining.
Written by AI from the articles below · updated Nov 12, 7:44 AM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
- PyTorch BlogOfficialPickEUROPA explains how it trained a 1T-parameter MoE on JUPITER with PyTorch
AIThe EUROPA consortium says it raised training throughput on the JUPITER supercomputer from 90 to about 1,300 tokens/s per GPU across 384 GH200 GPUs. The post describes the changes behind the gain, including FlexShard placement for the Muon optimizer, activation-checkpoint offloading to the Grace CPU, and running FSDP's all-gather on the compute stream.
Heat Trend
- Current comparable heat
- 6
- Peak comparable heat
- 10Nov 12, 8:00 AM ET
- Change in past 24 hours
- –
The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.