MooreThread open-sources Megatron-MUSA-Titan, focusing on trillion-parameter MoE model training for multi-thousand node clusters.

MooreThread open-sources Megatron-MUSA-Titan project, supporting trillion-parameter MoE model training for multi-thousand node clusters, covering pre-training and reinforcement learning scenarios, achieving high MFU and long-stable training. The project supports blockwise FP8 training, combining DeepEP and ACE asynchronous communication engines to accelerate token distribution and result merging, improving end-to-end training efficiency.