Tencent Hunyuan · Hugging Face new models·· 2 days ago
tencent/Hy4-base
tencent/Hy4-base
Brief overview
Hunyuan releases 770B parameter MoE model Hy4-base, supporting text and image understanding, with 49B parameters activated per token, including 78 layers (1 dense + 77 MoE), each MoE layer with 256 routing experts + 1 shared expert, activating top-8 routing experts + shared expert; built-in 3.3B parameter DFly2 module supports low-cost long-context reasoning, with a context window up to 1M tokens.
Source: Tencent Hunyuan · Hugging Face new models · huggingface.co