Anthropic's Fable 5.2 completes 41 percent of real freelance jobs, up from 2.5 percent a year ago

The progress shows how fast AI agents are catching up on practical work. When the benchmark launched about a year ago, the top system at the time, Manus, managed just 2.5 percent of tasks. Fable 5.2 now completes more than 16 times as many professional jobs.

The jump is also large compared to the most recent measurements. In July, Fable 5, an older model from the same family, was still in front at about 16 percent. That score was already a big step forward: the best agents had more than quadrupled the share of jobs they completed at professional quality within eight months. Four months later, Fable 5.2 hits 41.3 percent. That's roughly two and a half times Fable 5's July score. Its direct predecessor, Fable 5.1, also trails far behind at just under 18 percent, and Fable 5.2 completes more than twice as many jobs.

The next spots on the current leaderboard go to Anthropic's Opus 5.5 at 21.25 percent and OpenAI's GPT 6 Astra at 20.83 percent. Fable 5.2 works almost twice as successfully as the next-best systems on the market. Google's strongest model in the test, Gemini Argon 4, lands in fifth place at 17.08 percent, while older models like GPT-5.2 or Gemini 3 Pro sit below three percent.

Fable 5.2 breaks through in 3D models, audio, and architecture

Scale Labs said on X that Fable 5.2 makes its biggest gains in editing audio and music, designing architecture and interior spaces, and creating 3D models. The agent also handles long, complex projects that previously top-ranked models regularly failed.

Nearly 50 of Fable 5.2's successful completions were jobs no AI agent in the test had solved before. Scale Labs says the AI can still barely handle other tasks, especially industrial product design and animations. When AI agents fail in the test, they typically deliver amateur-quality work, incomplete results, broken files, or contradictory designs, according to the researchers.

Real jobs worth $144,000

Unlike conventional AI tests that often check only isolated skills, the RLI research paper covers complete work assignments. The dataset includes 240 completed projects from 358 verified professionals on the Upwork platform. Together, these jobs are worth about $144,000.

A single project costs an average of $632.60 and takes a human freelancer an average of just under 29 hours. Tasks range from video editing to CAD design, game development, and data analysis. To work on these jobs, AI agents get access to a virtual workspace with common professional software like Blender, GIMP, or Audacity, along with specialized AI tools for image, audio, and video generation.

Trained human experts manually check whether an AI agent completed a job successfully. They open the submitted files in the relevant professional software and compare the AI's output directly with the work the human freelancer delivered. CAIS's attempts to automate this labor-intensive review with another AI have so far failed. Because AI reviewers can barely operate the professional software themselves, they're fooled by submissions that look polished on the surface but are technically flawed, and they rate the models far too generously.