Skip to content
Live
Trending storyPast story

Embodied First Exam Released

1 reports1 outlets8 days agoUpdate

The story so far

AI overview

Hugging Face has released the first robot control evaluation framework for coding agents, Embodied First Exam, which includes 1,156 tasks across 35 sets on 47 robots, designed to evaluate the control capabilities of coding agents under physical world constraints. GPT-6 Astra in Codex achieved a success rate of 34% in 456 core tasks under standard settings. The framework provides three evaluation settings: standard settings simulate the real world, where agents only receive sensor observations and cannot reset the environment by default; the real-time variant keeps the environment running for agents to reason; the privileged setting allows agents to access the full simulator state and unlimited resets, used for searching oracles for training.

AI-written from the reports · Updated 7 days ago

Coverage timeline

Follow the coverage to see every side of the story.

11 Nov
  1. Hugging Face Daily Papers
    Embodied First Exam

    Embodied First Exam has been released, including 35 sets, 47 robots, and 1,156 tasks, to evaluate the control capabilities of coding agents under physical world constraints, with GPT-6 Astra in Codex achieving a success rate of 34% in 456 core tasks under standard settings.

Heat over time

Now
3
Peak
512 Nov, 11:00
24h change
–

Hover or tap the chart to see each hour; use the arrow keys on a keyboard.