Δτ-Bench 发布：推动智能体适应动态工具生态的能力评测
Δτ-Bench: Pushing Agents Toward a Moving Capability
Δτ-Bench 评测智能体适应动态工具生态的能力，发现当前最强模型仅达 42.3% 任务成功率。Δτ-Bench 包含 1,588 个跨 42 个领域的多轮任务和 6 个生态系统状态。Steward 通过增量技能维护和动态资源调配，在生态变化中实现更高的任务成功率。
Published on Oct 25
Authors:
,
,
,
,
,
,
,
Abstract
Δτ-Bench evaluates agent adaptability to evolving tool ecosystems, and Steward improves sustained success through incremental skill maintenance and dynamic provisioning.
In this paper, we introduce Δτ-Bench, an executable agent benchmark that models the structural evolution of large tool ecosystems. In Δτ-Bench, independently evolving services continually reshape how tasks can be accomplished: familiar tools may disappear or be replaced, while more efficient alternatives emerge amid overlapping functionality and noisy documentation. Against this moving capability frontier, we evaluate whether agents can proactively discover efficient tool-use paths and promptly revise their accumulated procedural knowledge, bringing together demands that existing benchmarks fail to examine. Δτ-Bench comprises 1,588 validated multi-turn tasks across 42 domains and six successive ecosystem states generated through capability-preserving transformations. Our evaluation shows that even the strongest evaluated model or harness achieves only 42.3% task success, while initial gains from existing skill-learning methods can not consistently translate into better adaptation as the ecosystem evolves. These findings motivate Steward, an ecosystem manager that jointly manages tools and procedural knowledge through incremental skill maintenance and dynamic provisioning during task execution. Experiments show that Steward sustains higher task success across ecosystem changes with lower skill-maintenance overhead, while improving both recovery from disrupted workflows and adoption of more efficient alternatives.
View arXiv page View PDF GitHub 3 Add to collection
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.34238 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.34238 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.34238 in a Space README.md to link it from this page.
Collections including this paper 1
来源: Hugging Face Daily Papers · huggingface.co