Continual Learning Exam: Benchmarking Long-Horizon RSI and Experiential Learning in LLM Agents
Continual Learning Exam: Benchmarking Long-Horizon RSI and Experiential Learning in LLM Agents
Continual Learning Exam (CLE) released, a benchmark including 120 real tasks, 600 progressive levels, and 89 professions, for evaluating the long-term continual learning and experience learning of LLM agents. Tested 17 cutting-edge configurations, the strongest system Claude Code + Opus 5 only completed 47.7% of the levels and 23.3% of the task steps.
🚀 Introducing Continual Learning Exam (CLE): Benchmarking Long-Horizon RSI and Experiential Learning in LLM Agents!
🧠 Can LLM agents truly learn from experience and continuously improve over time?
Most existing agent benchmarks focus on whether agents can solve individual tasks. But real-world intelligence requires more than task completion: agents should learn from past successes and failures, accumulate experience, and transfer acquired knowledge, strategies, and skills to future challenges.
To bridge this gap, we introduce Continual Learning Exam (CLE), a large-scale benchmark designed to evaluate long-horizon continual learning and experience-driven self-improvement in LLM agents.
🌟 Key Highlights:
🔹 Built with 50+ domain experts, covering 120 real-world professional tasks, 600 progressive levels, and 89 occupations across 12 domains.
🔹 Realistic professional environments, featuring ~24K real-world files, 19 stateful applications, and 16K+ human-authored evaluation criteria.
🔹 Game-like Task Ladders, where agents must complete each level to unlock the next. Later levels require agents to reuse and build upon knowledge, strategies, and skills acquired through earlier experience.
📊 Key Findings:
We evaluated 17 frontier agent configurations. Even the strongest system, Claude Code + Opus 5, achieves only:
• 47.7% level completion rate
• 23.3% full Task Ladder completion rate
These results reveal a substantial gap between solving individual tasks and achieving genuine long-term continual learning.
We hope CLE can serve as a realistic and verifiable benchmark to advance research on Agent Continual Learning, Memory, Skill Learning, and Recursive Self-Improvement (RSI).
💡 Beyond Task Solving: Towards Continual Learning and Experience-Driven Self-Improvement.
We welcome feedback, discussions, and collaborations from the community!
Source: Hugging Face Daily Papers · huggingface.co