发布 WeMM-Video 开源多模态视频理解模型家族
WeMM-Video: Pushing the Frontiers of Open-Source Omni-Modal Video Understanding
WeMM-Video 是一个包含 2B、4B、9B 和 35B-A3B 四个版本的中英双语开源多模态基础模型家族,通过混合思维模式和大规模训练实现文本、图像、音频和视频的统一处理,支持非思维指令模式和思维推理模式。
Published on Nov 19
Authors:
,
,
,
,
,
,
,
,
,
,
Abstract
WeMM-Video is a family of Chinese-English bilingual omni-modal foundation models that unify text, image, audio, and video processing for joint audio-visual perception and reasoning, using hybrid thinking modes and large-scale training to achieve strong open-source video understanding results.
Video understanding, particularly with accompanying audio, remains a central challenge in multimodal intelligence. In this report, we introduce WeMM-Video, a family of Chinese-English bilingual omni-modal foundation models spanning 2B, 4B, 9B, and 35B-A3B variants. Centering on joint audio-visual perception and reasoning, WeMM-Video unifies text, image, audio, and video processing within a hybrid-thinking paradigm, seamlessly supporting both a non-thinking instruct mode for fast, direct responses and a thinking mode for explicit step-by-step reasoning. The models are trained on a large-scale multimodal corpus of approximately 1.2 trillion tokens through progressive pre-training and a multi-stage post-training pipeline incorporating multi-teacher OPD, anti-repetition MrDPO, and specialist RL. Evaluated across more than 120 benchmarks, WeMM-Video demonstrates superior performance in omni-modal video understanding and reasoning, while maintaining highly competitive capabilities across general audio, silent video, image, and text tasks. Specifically, in omni-modal video evaluations, our flagship WeMM-Video-35B-A3B establishes a new open-source state of the art, substantially outperforming proprietary models such as Qwen3.5-Omni-Flash and Seed2.0-Mini-0428 while narrowing the gap with the strong Gemini-3.0-Flash, and our compact WeMM-Video-9B surpasses the larger open-source Qwen3-Omni-30B-A3B-Instruct. We publicly release the WeMM-Video family along with our evaluation and reproduction pipeline to foster further research and development in the open-source community.
View arXiv page View PDF GitHub 5 Add to collection
Get this paper in your agent:
hf papers read 2611.26409
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 20
Image-Text-to-Text • 36B • Updated about 2 hours ago • 7tencent/WeMM-Video-35BA3B
Image-Text-to-Text • 10B • Updated about 2 hours ago • 5tencent/WeMM-Video-9B
Image-Text-to-Text • 3B • Updated about 2 hours ago • 5tencent/WeMM-Video-2B
Image-Text-to-Text • 36B • Updated about 2 hours ago • 3tencent/WeMM-Video-35BA3B-FP8
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2611.26409 in a dataset README.md to link it from this page.
Spaces citing this paper 1
Collections including this paper 1
来源: Hugging Face Daily Papers · huggingface.co