跳到正文
实时
Hugging Face Daily Papers·· 1 天前精选73

V-JEPA-LM 开源视频语言模型 Llama-V-JEPA-LM 发布

V-JEPA-LM: Video-Language Models That Read Dynamics Beyond Static, Text-Aligned Representations

要点速览

研究团队发布开源视频语言模型 Llama-V-JEPA-LM,将冻结的 V-JEPA 2.1 世界模型编码器与 Qwen3 / Llama(1B–8B)结合,采用单阶段训练和反事实监督:同一片段及其变换副本(倒放、镜像、改变时序)配同一问题但不同答案,使模型只能依据变化本身作答。

Why it matters

在视频动态理解这一薄弱环节上,该工作用反事实监督把冻结的 V-JEPA 2.1 世界模型接入 Qwen3/Llama,并给出与 Meta 视频 LLM 的数据和 token 用量对比。

正文 · 原文

We release Llama-V-JEPA-LM, open video-language models that join a frozen V-JEPA 2.1 world-model encoder to Qwen3 / Llama (1B–8B), trained in a single stage with counterfactual supervision: a clip and its transformed copy (reversed, mirrored, re-timed) get the same question with different answers, so only the change itself can explain the answer.

  • Leads every size class on 11 video benchmarks (4B: 72.14 avg), strongest on dynamics (TOMATO, Vinoground, SSv2)
  • Beats Meta's V-JEPA 2 / 2.1 video LLMs with 1/20 of the training data and 1/9 of the visual tokens
  • 4.53M public samples, 32 visual tokens per frame, one training stage

All 6 checkpoints, code and a live demo are open.

来源: Hugging Face Daily Papers · huggingface.co