Baidu Wenxin has open-sourced the ERNIE 5.5 Flash Base base model, with a total parameter of 280B.
Baidu Wenxin has released ERNIE-5.5-Flash-Base and opened the ERNIE 5.5 Flash pre-trained endpoint weights. This is a sparse MoE language model with a total parameter of 280B and 14B activation per token. It was pre-trained from scratch on 33.5T tokens and supports fine-tuning and inference with a context size of up to 1M tokens.
The original text provides the parameters, training data size, and open-source weights of the 280B sparse MoE base, allowing readers to judge its adaptability as a fine-tuning starting point.
Same story3 other outlets also reported this
- ITHome · AI··First·Reading now
Baidu Wenxin has open-sourced the ERNIE 5.5 Flash Base base model, with a total parameter of 280B.
- The Decoder··+9 h 36 min
Baidu released the open-source ERNIE 5.5 Flash Base large model
Adds:The Decoder mentions the pre-training covers 3350 trillion tokens.
- MarkTechPost··+13 h 12 min
Baidu released ERNIE 5.5 Flash Base: a 280B MoE base model, activating 14B parameters per token.
In order of publication · later reports are added as they appear · Compared by AI from each outlet’s title and summary
Source: ITHome · AI · ithome.com