Embodied First Exam Released
The story so far
Hugging Face has released the first robot control evaluation framework for coding agents, Embodied First Exam, which includes 1,156 tasks across 35 sets on 47 robots, designed to evaluate the control capabilities of coding agents under physical world constraints. GPT-6 Astra in Codex achieved a success rate of 34% in 456 core tasks under standard settings. The framework provides three evaluation settings: standard settings simulate the real world, where agents only receive sensor observations and cannot reset the environment by default; the real-time variant keeps the environment running for agents to reason; the privileged setting allows agents to access the full simulator state and unlimited resets, used for searching oracles for training.
AI-written from the reports · Updated 7 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- Hugging Face Daily PapersEmbodied First Exam
Embodied First Exam has been released, including 35 sets, 47 robots, and 1,156 tasks, to evaluate the control capabilities of coding agents under physical world constraints, with GPT-6 Astra in Codex achieving a success rate of 34% in 456 core tasks under standard settings.
Heat over time
- Now
- 3
- Peak
- 512 Nov, 11:00
- 24h change
- –
Hover or tap the chart to see each hour; use the arrow keys on a keyboard.