Skip to content
Live
Hugging Face Daily Papers·· yesterdayFeatured73

V-JEPA-LM: Video-Language Models That Read Dynamics Beyond Static, Text-Aligned Representations

V-JEPA-LM: Video-Language Models That Read Dynamics Beyond Static, Text-Aligned Representations

Brief overview

The research team released the open-source video language model Llama-V-JEPA-LM, combining the frozen V-JEPA 2.1 world model encoder with Qwen3 / Llama (1B–8B), using single-stage training and counterfactual supervision: the same segment and its transformed copies (reversed, mirrored, changed temporally) are paired with the same question but different answers, making the model answer only based on the changes themselves.

Why it matters

On the weak point of video dynamic understanding, this work uses counterfactual supervision to connect the frozen V-JEPA 2.1 world model to Qwen3/Llama, and provides a comparison of data and token usage with Meta video LLM.

Article

We release Llama-V-JEPA-LM, open video-language models that join a frozen V-JEPA 2.1 world-model encoder to Qwen3 / Llama (1B–8B), trained in a single stage with counterfactual supervision: a clip and its transformed copy (reversed, mirrored, re-timed) get the same question with different answers, so only the change itself can explain the answer.

  • Leads every size class on 11 video benchmarks (4B: 72.14 avg), strongest on dynamics (TOMATO, Vinoground, SSv2)
  • Beats Meta's V-JEPA 2 / 2.1 video LLMs with 1/20 of the training data and 1/9 of the visual tokens
  • 4.53M public samples, 32 visual tokens per frame, one training stage

All 6 checkpoints, code and a live demo are open.

Source: Hugging Face Daily Papers · huggingface.co