V-JEPA-LM 开源视频语言模型 Llama-V-JEPA-LM 发布
V-JEPA-LM: Video-Language Models That Read Dynamics Beyond Static, Text-Aligned Representations
研究团队发布开源视频语言模型 Llama-V-JEPA-LM,将冻结的 V-JEPA 2.1 世界模型编码器与 Qwen3 / Llama(1B–8B)结合,采用单阶段训练和反事实监督:同一片段及其变换副本(倒放、镜像、改变时序)配同一问题但不同答案,使模型只能依据变化本身作答。
在视频动态理解这一薄弱环节上,该工作用反事实监督把冻结的 V-JEPA 2.1 世界模型接入 Qwen3/Llama,并给出与 Meta 视频 LLM 的数据和 token 用量对比。
We release Llama-V-JEPA-LM, open video-language models that join a frozen V-JEPA 2.1 world-model encoder to Qwen3 / Llama (1B–8B), trained in a single stage with counterfactual supervision: a clip and its transformed copy (reversed, mirrored, re-timed) get the same question with different answers, so only the change itself can explain the answer.
- Leads every size class on 11 video benchmarks (4B: 72.14 avg), strongest on dynamics (TOMATO, Vinoground, SSv2)
- Beats Meta's V-JEPA 2 / 2.1 video LLMs with 1/20 of the training data and 1/9 of the visual tokens
- 4.53M public samples, 32 visual tokens per frame, one training stage
All 6 checkpoints, code and a live demo are open.
来源: Hugging Face Daily Papers · huggingface.co