Skip to content
Live
Trending storyPast story

OpenDS-World benchmark released

1 reports1 outlets27 days agoUpdate

The story so far

AI overview

qwen released the OpenDS-World benchmark to evaluate autonomous deep search agents in real network environments, showing a significant gap between cutting-edge models and human performance. Among 12 cutting-edge models, the best model achieved an accuracy of 39.0%, while the accuracy of 1000 human evaluations was 21.1%. Analysis indicates that research strategies vary among models, and tool coordination and long-term search management are major challenges. OpenDS-World introduces a real-time network benchmark to evaluate autonomous multi-step deep search agents, revealing a significant gap between cutting-edge models and human performance.

AI-written from the reports · Updated 27 days ago

Coverage timeline

Follow the coverage to see every side of the story.

23 Oct
  1. Hugging Face Daily Papers
    Deep Search in Real-World Web Environments

    OpenDS-World introduces a real-time network benchmark to evaluate autonomous multi-step deep search agents, revealing a significant gap between cutting-edge models and human performance. Among 12 cutting-edge models, the best model achieved an accuracy of 39.0%, while 1000 human evaluations reached an accuracy of 21.1%. Analysis shows that research strategies vary among models, and tool coordination and long-term search management remain major challenges.