PDRVLA: A Vision-Language-Action Model with Physical Dynamics Representations
PDRVLA: A Vision-Language-Action Model with Physical Dynamics Representations
PDRVLA improves robot action generation by integrating a lightweight physical dynamics module that predicts future visual features under the supervision of a physical world model, avoiding latency caused by online video generation. PDRVLA achieves success rates of 99.3% (LIBERO), 89.9% (LIBERO-Plus), 93.6% (RoboTwin~2.0), and 81.5% (SimplerEnv) on four simulation benchmarks.
Published on Oct 27
·
Submitted by
on Oct 28
Authors:
,
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
PDRVLA improves robot action generation by integrating a lightweight physical dynamics module that predicts future visual features with world-model supervision, avoiding online video generation latency.
Vision--Language--Action (VLA) models generate robot actions from visual observations and language instructions, but their understanding of physical dynamics remains limited. Video world models can capture scene dynamics, yet generating future videos introduces high latency into the control loop. We introduce PDRVLA, whose lightweight Physical Dynamics Module extracts physical tokens from vision--language features to condition action generation without online video generation. The module learns through direct prediction of future RGB, depth, and optical-flow features, with auxiliary supervision through a frozen pretrained video world model. Training relies entirely on publicly available data, expanded through human-to-robot synthesis, subtask annotation, and embodied chain-of-thought (ECoT) annotation. Across four simulation benchmarks, PDRVLA achieves success rates of 99.3% on LIBERO, 89.9% on LIBERO-Plus, 93.6% on RoboTwin~2.0, and 81.5% on SimplerEnv. Ablations show improved future prediction with world-model supervision and evaluate how subtask conditioning, future object-pose supervision, and ECoT training affect action success.
View arXiv page View PDF Add to collection
Get this paper in your agent:
hf papers read 2610.37381
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.37381 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.37381 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.37381 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
Source: Hugging Face Daily Papers · huggingface.co