Skip to content
Live
Tencent Hunyuan · Hugging Face new models·· 22 days ago

tencent/Hy4

tencent/Hy4

Brief overview

Hunyuan has released the new generation 770B parameter MoE model Hunyuan Hy4, supporting text, image, and video understanding, with a language backbone of 49B parameters/Token, including 78 layers (1 dense layer + 77 MoE layers), each MoE layer with 256 routing experts + 1 shared expert, selecting Top-8 routing experts + shared expert when activated; the built-in 3.3B parameter DFly2 module supports low-cost long-context reasoning, with a context window up to 1M Token.

Source: Tencent Hunyuan · Hugging Face new models · huggingface.co