ChronoReplay: One Answer Interface Is Not Enough for Streaming Spatial Intelligence
ChronoReplay: One Answer Interface Is Not Enough for Streaming Spatial Intelligence
ChronoReplay training framework organizes heterogeneous spatiotemporal supervision into time-ordered interaction segments, preserves specific task answer formats, and balances learning from different supervision signals, enabling a single model to learn complementary spatiotemporal capabilities while maintaining temporal consistency. It achieves improvements on OVO-S-Bench, OST-Bench, and UCS-Bench benchmarks, with a 13.41-point increase on OST-Bench, while optimizing streaming inference performance through adaptive frame selection.
ChronoReplay is a training framework designed to improve the spatial reasoning capabilities of streaming vision-language models (VLMs).
Existing spatial VLMs often specialize in specific answer formats, such as categorical spatial relations, metric distances, or motion trajectories. This specialization, which we call interface lock-in, limits generalization across different spatial reasoning tasks.
To address this challenge, ChronoReplay organizes heterogeneous spatial supervision into chronologically ordered episodes, preserves task-specific answer formats, and balances learning across supervision families. This enables a single model to learn complementary spatial capabilities while maintaining temporal consistency.
Key Highlights:
Unified Spatial Learning: Supports categorical, metric, and trajectory-based spatial reasoning within a single model.
Chronological Replay: Preserves temporal dependencies by organizing training examples into ordered interaction episodes.
Family-Balanced Training: Improves cross-task generalization by balancing heterogeneous supervision signals.
Strong Benchmark Performance: Achieves consistent improvements across OVO-S-Bench, OST-Bench, and UCS-Bench, including a 13.41-point improvement on OST-Bench.
Efficient Streaming Inference: Adaptive frame selection further improves performance while reducing visual processing costs.
ChronoReplay demonstrates that effective streaming spatial intelligence requires preserving complementary spatial information across tasks, learning from temporally structured experiences, and selecting relevant visual evidence at inference time.
Source: Hugging Face Daily Papers · huggingface.co