Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models
VLAct proposes representation-centric continued pre-training for Vision-Language-Action models by distilling robot trajectories into generalized knowledge. It addresses failures in naive methods by using shallow-layer protection, multi-head co-supervision, and unified action representations. Results show VLAct significantly outperforms baselines on benchmarks like LIBERO-Plus and demonstrates superior data efficiency.