StreamEval: 评估交互式流式视频生成模型的基准
StreamEval: Evaluating Interactive Streaming Video Generation Models
StreamEval 基准测试了交互式流式视频生成模型在五个任务上的表现，发现可靠的请求执行和持续的状态一致性仍是主要挑战。该基准包含 300 个脚本化测试用例，涵盖角色对话、角色表演、世界生成、指令编辑和参考编辑任务，通过四个评估维度（任务执行、状态一致性、视觉保真度和视听协调）进行评分。
Abstract
StreamEval benchmarks interactive streaming video generation across five tasks, revealing that reliable request execution and sustained state consistency remain major challenges despite high visual quality.
As streaming video generation advances, enabling real-time interaction with generated video is becoming increasingly important. We introduce StreamEval, a unified benchmark for interactive streaming video generation spanning five tasks: avatar dialogue, avatar performance, world generation, instruction editing, and reference editing. StreamEval comprises 300 scripted test cases lasting 15-300s, each specifying initial assets, incrementally arriving multimodal requests, and explicit requirements for state changes and content preservation. Performance is assessed along four evaluation dimensions: task execution, state consistency, visual fidelity, and audiovisual coordination, instantiated through ten task-specific metrics on a 0-100 scale. Evaluating 37 distinct models across 300 test cases, we find that high visual quality and identity retention do not necessarily coincide with reliable request execution or sustained state consistency. In particular, request execution remains challenging: the highest task-specific scores are 36.90 for Following among seven avatar-performance models, 25.93 for Following in text-conditioned world generation, and 15.00 for Edit Success among three reported reference-editing models. These findings identify reliable request execution under timing and preservation constraints as a central challenge for advancing omni-modal real-time interactive systems. The leaderboard is available at https://streameval.ai
View arXiv page View PDF Project page GitHub 129 Add to collection
Get this paper in your agent:
hf papers read 2610.28402
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.28402 in a model README.md to link it from this page.
Datasets citing this paper 1
StreamEval/StreamEval
Viewer • Updated about 3 hours ago • 1.32k • 57 • 6
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.28402 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
来源: Hugging Face Daily Papers · huggingface.co