Skip to content
Live
Hugging Face Daily Papers·· yesterday65

VibePrevisBench: Evaluating Agentic Coding for 4D Scene Generation in Blender

VibePrevisBench: Evaluating Agentic Coding for 4D Scene Generation in Blender

Brief overview

VibePrevisBench is a benchmark for 4D scene reconstruction that allows agents to transform briefs into executable 4D scenes and render agent videos, containing 242 tasks, scored through rule-based scene checks and screen checks based on visual language models, analysis shows multimodal understanding ability distinguishes agents, all agents master basic operations but fail to handle complex requests and fail to correct rendering errors.

Article

Published on Nov 11

#3 Paper of the day

Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

Controllable real-world video generation models can condition on a rendered graybox proxy video, which fixes layout, motion and camera. Coding agents can build such videos by reconstructing a 4D scene in Blender, yet no benchmark checks both these videos and the long, multi-shot scenes behind them. We introduce VibePrevisBench, a benchmark for 4D scene reconstruction: an agent turns a brief into an executable 4D scene and renders its proxy video. Its 242 tasks span four input kinds, including scripts, photos, continuations, and reference videos. Each delivery is scored along two evaluation dimensions. Rule-based Scene Check re-runs the code and compares the exported 4D scene with hidden ground truth facts in nine dimensions. VLM-based Screen Check asks a VLM judge whether the facts that pass can be recognized in the proxy video. We analyze the evaluation results and trajectories of 11 frontier coding agents and identify three key observations. First, multimodal understanding separates the agents, especially in understanding space and camera motion. Second, all agents seem to already master basic camera work and staging, and fail at challenging requests. Third, agents inspect their own renders but fail to correct their errors. Agents thus need better visual understanding of space and camera, explicit checks for challenging requests, and item-by-item verification of renders against the brief.

View arXiv page View PDF Project page GitHub Add to collection

Get this paper in your agent:

hf papers read 2611.15789

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.15789 in a model README.md to link it from this page.

Datasets citing this paper 1

Kling-Keye/vibeprevisbench

Viewer •

Updated about 4 hours ago

•

242

•

2.04k

•

3

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.15789 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

Source: Hugging Face Daily Papers · huggingface.co