跳到正文
实时
Hugging Face Daily Papers·· 10 小时前

发布 WeMM-Video 开源多模态视频理解模型家族

WeMM-Video: Pushing the Frontiers of Open-Source Omni-Modal Video Understanding

要点速览

WeMM-Video 是一个包含 2B、4B、9B 和 35B-A3B 四个版本的中英双语开源多模态基础模型家族,通过混合思维模式和大规模训练实现文本、图像、音频和视频的统一处理,支持非思维指令模式和思维推理模式。

正文

Published on Nov 19

#2 Paper of the day

Authors:

,

,

,

,

,

,

,

,

,

,

Abstract

WeMM-Video is a family of Chinese-English bilingual omni-modal foundation models that unify text, image, audio, and video processing for joint audio-visual perception and reasoning, using hybrid thinking modes and large-scale training to achieve strong open-source video understanding results.

Video understanding, particularly with accompanying audio, remains a central challenge in multimodal intelligence. In this report, we introduce WeMM-Video, a family of Chinese-English bilingual omni-modal foundation models spanning 2B, 4B, 9B, and 35B-A3B variants. Centering on joint audio-visual perception and reasoning, WeMM-Video unifies text, image, audio, and video processing within a hybrid-thinking paradigm, seamlessly supporting both a non-thinking instruct mode for fast, direct responses and a thinking mode for explicit step-by-step reasoning. The models are trained on a large-scale multimodal corpus of approximately 1.2 trillion tokens through progressive pre-training and a multi-stage post-training pipeline incorporating multi-teacher OPD, anti-repetition MrDPO, and specialist RL. Evaluated across more than 120 benchmarks, WeMM-Video demonstrates superior performance in omni-modal video understanding and reasoning, while maintaining highly competitive capabilities across general audio, silent video, image, and text tasks. Specifically, in omni-modal video evaluations, our flagship WeMM-Video-35B-A3B establishes a new open-source state of the art, substantially outperforming proprietary models such as Qwen3.5-Omni-Flash and Seed2.0-Mini-0428 while narrowing the gap with the strong Gemini-3.0-Flash, and our compact WeMM-Video-9B surpasses the larger open-source Qwen3-Omni-30B-A3B-Instruct. We publicly release the WeMM-Video family along with our evaluation and reproduction pipeline to foster further research and development in the open-source community.

View arXiv page View PDF GitHub 5 Add to collection

Get this paper in your agent:

hf papers read 2611.26409

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 20

tencent/WeMM-Video-35BA3B

Image-Text-to-Text •

36B •

Updated about 2 hours ago

•

7

tencent/WeMM-Video-9B

Image-Text-to-Text •

10B •

Updated about 2 hours ago

•

5

tencent/WeMM-Video-2B

Image-Text-to-Text •

3B •

Updated about 2 hours ago

•

5

tencent/WeMM-Video-35BA3B-FP8

Image-Text-to-Text •

36B •

Updated about 2 hours ago

•

3

Browse 20 models citing this paper

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.26409 in a dataset README.md to link it from this page.

Spaces citing this paper 1

Collections including this paper 1

来源: Hugging Face Daily Papers · huggingface.co