ReTrace框架：通过目标导向行动回放揭示计算机使用智能体的恢复行为
Lost in Recovery: Demystifying Recovery Behavior of Computer-Use Agents via Goal-Oriented Action Replay
ReTrace框架通过目标导向行动回放和受控错误注入的形式化过渡一致性恢复标准，首次系统性地研究了计算机使用智能体的自主模型恢复行为，发现理想恢复能解锁模型潜在任务完成能力，当前智能体未充分利用其潜在恢复能力，且准确模型恢复可带来可衡量的端到端增益，证明恢复是构建可靠计算机使用智能体的基本能力。
Published on Oct 20
Authors:
,
,
,
,
Abstract
ReTrace is a framework that evaluates recovery behavior in computer-use agents by formalizing transition-consistent recovery and demonstrating that autonomous model-based recovery improves long-horizon reliability.
Recent advances in large (vision) language models have enabled computer-use agents that automate digital workflows via fine-grained keyboard and mouse operations. However, such agents remain fragile in long-horizon settings, where errors accumulate and executed actions leave unintended side effects, making the ability to recover from failures critical. Yet, existing remedies remain inadequate: environment resets are infeasible in proprietary settings, action replay is prohibitively expensive over long horizons, and API-based rollback is limited to web environments, leaving model-based recovery largely unexplored. To address this gap, we present ReTrace, a scalable evaluation framework for systematically studying recovery behavior in computer-use agents. We formalize Transition-Consistent Recovery: identical downstream actions from the recovered and target states must yield identical outcomes. We operationalize this criterion through goal-oriented action replay following controlled error injection. Using ReTrace, we present the first systematic study of autonomous, model-based recovery, revealing that (i) ideal recovery can unlock model's latent task completion capabilities, (ii) current computer-use agents fail to exploit their latent recovery capabilities, and (iii) accurate model-based recovery yields measurable end-to-end gains. Together, these findings establish recovery as a fundamental capability for building reliable computer-use agents. Our code and dataset is available at https://github.com/BangProx/ReTRACE.
View arXiv page View PDF GitHub 3 Add to collection
Get this paper in your agent:
hf papers read 2610.27954
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.27954 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.27954 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.27954 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
来源: Hugging Face Daily Papers · huggingface.co