EUROPA Union trains giant MoE model
The story so far
The EUROPA Union successfully trained a MoE model with 951.9B parameters on the JUPITER supercomputer using PyTorch and TorchTitan, increasing the throughput of a single GPU from 90 to approximately 1,300 tokens/s, utilizing 384 NVIDIA GH200 GPUs. This event's core is the EUROPA Union achieving large-scale model training through technical means, demonstrating its progress in the field of high-performance computing.
AI-written from the reports · Updated 2 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- PyTorch BlogEUROPA Union trains a 1T parameter MoE model using PyTorch and TorchTitan on the JUPITER supercomputer
The EUROPA Union successfully trained a MoE model with 951.9B parameters on the JUPITER supercomputer using PyTorch and TorchTitan, increasing single GPU throughput from 90 to approximately 1,300 tokens/s with 384 NVIDIA GH200 GPUs.
Heat over time
- Now
- 3
- Peak
- 1012 Nov, 21:00
- 24h change
- -50%
Hover or tap the chart to see each hour; use the arrow keys on a keyboard.