ChronoReplay 训练框架提升流式视觉语言模型的时空推理能力
ChronoReplay: One Answer Interface Is Not Enough for Streaming Spatial Intelligence
ChronoReplay训练框架通过将异构时空监督组织成按时间排序的交互片段、保留特定任务答案格式、平衡不同监督信号学习，使单一模型能学习互补时空能力并保持时间一致性，在 OVO-S-Bench、OST-Bench 和 UCS-Bench 基准测试中均取得提升，OST-Bench 提高 13.41 分，同时通过自适应帧选择优化流式推理性能。
ChronoReplay is a training framework designed to improve the spatial reasoning capabilities of streaming vision-language models (VLMs).
Existing spatial VLMs often specialize in specific answer formats, such as categorical spatial relations, metric distances, or motion trajectories. This specialization, which we call interface lock-in, limits generalization across different spatial reasoning tasks.
To address this challenge, ChronoReplay organizes heterogeneous spatial supervision into chronologically ordered episodes, preserves task-specific answer formats, and balances learning across supervision families. This enables a single model to learn complementary spatial capabilities while maintaining temporal consistency.
Key Highlights:
Unified Spatial Learning: Supports categorical, metric, and trajectory-based spatial reasoning within a single model.
Chronological Replay: Preserves temporal dependencies by organizing training examples into ordered interaction episodes.
Family-Balanced Training: Improves cross-task generalization by balancing heterogeneous supervision signals.
Strong Benchmark Performance: Achieves consistent improvements across OVO-S-Bench, OST-Bench, and UCS-Bench, including a 13.41-point improvement on OST-Bench.
Efficient Streaming Inference: Adaptive frame selection further improves performance while reducing visual processing costs.
ChronoReplay demonstrates that effective streaming spatial intelligence requires preserving complementary spatial information across tasks, learning from temporally structured experiences, and selecting relevant visual evidence at inference time.
来源: Hugging Face Daily Papers · huggingface.co