UserArena: Released a long-term shopping intelligent agent interaction benchmark
The story so far
UserArena introduced an e-commerce interaction benchmark containing 64,729 session trajectories, covering 11,010 users, 566,102 items, and 13 action types, supporting five-level evaluation. The On-Policy Trajectory Distillation technique was proposed to improve simulator fidelity, experiments showing it performs best under the UserArena evaluation protocol, closest to offline simulation effects of production strategies.
AI-written from the reports · Updated 24 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- Hugging Face Daily PapersUserArena: Benchmarking Interactive User Simulation for Long-Horizon Shopping Agents
UserArena introduces an e-commerce interaction benchmark containing 64,729 session trajectories, covering 11,010 users, 566,102 items, and 13 action types, supporting five-level evaluation; proposes On-Policy Trajectory Distillation technology to improve simulator fidelity, experiments show it works best under the UserArena evaluation protocol, closest to offline simulation effects of production strategies.