Skip to content
Live
Trending storyPast story

Hugging Face Daily Papers: Research on Continual Pre-training of Embodied Agents

1 reports1 outlets28 days agoUpdate

The story so far

AI overview

Hugging Face Daily Papers reported on research involving continuous pre-training (CPT) on three dimensions: data scale, diversity, and model size, which improved the learning ability of embodied intelligent agents in virtual worlds but had limitations in long-distance world generalization. The study tested 120 configurations covering parameters from 0.8B to 27B, data budgets from 50 to 1600 hours, and five data mixes. It found that increased data volume improved internal performance, diversity enhanced related world transfer, and larger model sizes improved data efficiency but led to poorer long-distance generalization.

AI-written from the reports · Updated 27 days ago

Coverage timeline

Follow the coverage to see every side of the story.

22 Oct
  1. Hugging Face Daily Papers
    Scaling in Games: Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds

    Research has improved the learning ability of embodied agents in virtual worlds through continuous pre-training (CPT) across three dimensions: data scale, diversity, and model size, but there are limitations in long-distance world generalization. The study tested 120 configurations covering parameters from 0.8B to 27B, data budgets from 50 to 1600 hours, and five data mixtures. It was found that increasing data volume improves internal performance, diversity enhances related world transfer, and larger model sizes improve data efficiency but result in worse long-distance generalization.