Prime Intellect released Prime Mega MoE
The story so far
Prime Intellect has released Prime Mega MoE, a trainable fusion MoE core, made usable for training by increasing rapid overlapping backward pass. In Prime Intellect's prime-rl trainer, using the full training steps of the Qwen3 MoE model has boosted speeds by up to 34%, with a 55% increase compared to TorchTitan. This technology supports NVIDIA Hopper (blockwise FP8) and Blackwell (MXFP8) GPUs, and supports BF16 on both. The release data uses 8×H200 nodes, configured with 32 to 128 GPUs.
AI-written from the reports · Updated 20 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- MarkTechPostPrime Intellect Releases Prime Mega MoE: Trainable Fused MoE Kernel, 3.11x Faster Forward Pass
Prime Intellect released Prime Mega MoE, adding fast overlapping reverse transmission on the basis of DeepSeek's integrated Mega MoE core, making it available for training.