Synced·Added· 3 days ago65
The father of LSTM leads, KAUST, HKU, Huawei, and others release a review on language model architecture.
Brief overview
Architectures such as GQA, MLA, and linear attention networks continue to evolve, DeepSeek introduces fine-grained MoE, this review is led by the father of LSTM, jointly completed by institutions such as KAUST, HKU, and Huawei.
Source: Synced · jiqizhixin.com