Wizard: Manipulate Any World in Real Time via Agentic Control

Brief overview

Wizard decouples character animation, scene control, and game logic from visual generation, allowing intelligent agents to control joint skeletons and scene geometry through structured visual agents, achieving real-time interaction at 24 FPS, zero-shot generalization to unseen scenes, characters, and actions. The team built 50.3 hours of aligned agent-video pairs, including 750 scenes and 178 characters, supporting character adaptive masking, causal adaptation, and few-step distillation techniques.

Article

🪄 Wizard: Manipulate Any World in Real Time via Agentic Control!

Existing game world models produce realistic visuals but remain constrained by familiar environments and predefined motions. Control largely focuses on locomotion, with limited support for articulated actions. Can a world model follow an unseen character swinging a sword, spreading its wings, or transforming mid-battle?

Wizard decouples character animation, scene manipulation, and gameplay logic from visual generation. Agents control articulated skeletons and scene geometry through structured visual proxies, while explicit world states support adaptive boss behavior and mechanics-consistent combat.

We construct 50.3 hours of aligned proxy–video pairs spanning 750 scenes and 178 characters. Character-aware adaptive masking preserves reference appearance, while causal adaptation and few-step distillation enable interaction at 24 FPS, with zero-shot generalization to unseen scenes, characters, and motions.

From world exploration and character summoning to transformation and scene manipulation, Wizard gives players greater freedom to create and control their own game worlds.

Code coming soon!