NEO-VLA Framework Released
The story so far
Hugging Face released NEO-VLA framework, which eliminates fixed visual encoders by allowing action strategies to directly access deep visual representations in the network, achieving stronger robustness in various manipulation scenarios and distribution offsets. The framework uses lightweight convolutional layers to tokenize images and allows visual representations to evolve with language in the backbone network, enabling action experts to obtain information from different depth visual representations conditioned by instructions. Layer exploration experiments show that action key signals often reach their peak before the final visual representation is formed.
AI-written from the reports · Updated 13 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- Hugging Face Daily PapersFrom Pixels to Actions - Towards Native Vision-Language-Action Policies
NEO-VLA framework eliminates fixed visual encoders by allowing action strategies to directly access deep visual representations in the network, achieving stronger robustness in various manipulation scenarios and distribution shifts. The framework uses lightweight convolutional layers to tokenize images and allows visual representations to evolve together with language in the backbone network, enabling action experts to obtain information from different depths of instruction-conditioned visual representations. Layer exploration experiments show that action key signals often reach their peak before the final visual representation is formed.