HiFOR：一种用于视觉语言行动模型的层级预见框架
HiFOR: Hierarchical Foresight with Action-Guided World Grounding for Vision-Language-Action Model
HiFOR 通过层级预测未来状态并检索相关未来信息来指导行动生成，提升机器人操作能力。该框架通过层级依赖掩码（HDM）联合预测短期状态演变、语义里程碑和最终目标状态，建立跨时间范围依赖关系；结构感知潜在世界建模沿时间、视角和空间维度组织未来潜在表示，促进细粒度检索；行动引导分解世界检索根据去噪过程中的行动表示自适应检索相关未来信息，为行动生成提供针对性指导。
Published on Oct 29
Authors:
,
,
,
,
,
Abstract
HiFOR improves robotic manipulation by hierarchically predicting future states and retrieving relevant future information to guide action generation.
Vision-Language-Action (VLA) models have demonstrated promising capabilities in robotic manipulation by integrating visual perception, language understanding, and action generation. Recent advances have further incorporated future prediction to enhance the understanding of environmental dynamics and task progression. However, existing methods often model short-term dynamics and long-term task objectives separately, while directly incorporating predicted futures into action policies without sufficiently considering their relevance to current action requirements. To address these limitations, we propose HiFOR, a hierarchical foresight framework with action-guided world retrieval. Specifically, Hierarchical Foresight Prediction jointly predicts short-term state evolution, semantic milestones, and final goal states through a Hierarchical Dependency Mask (HDM), establishing dependencies across temporal horizons. Building upon these predictions, Structure-Aware Latent World Modeling organizes future latent representations along the horizon, view, and spatial dimensions to facilitate fine-grained retrieval. Furthermore, Action-Guided Factorized World Retrieval adaptively retrieves relevant future information from the structured latent world based on evolving action representations during denoising, providing targeted guidance for action generation. Extensive experiments on two simulation benchmarks and six real-world robotic manipulation tasks demonstrate the effectiveness of our approach.
View arXiv page View PDF Add to collection
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.40373 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.40373 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.40373 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
来源: Hugging Face Daily Papers · huggingface.co