Direction-Scale Decomposition in Action Representation: Rethinking What to Tokenize for Vision-Language-Action Models
cs.CV, cs.RO
Submitted: 2026-09-24
Updated: 2026-09-24
Project page: https://vla-dsd.github.io
License: http://creativecommons.org/licenses/by/4.0/
The gist: Action representation plays a central role in discrete-token vision-language-action (VLA) learning but remains underexamined.
Terminology
Abstract
Action representation plays a central role in discrete-token vision-language-action (VLA) learning but remains underexamined. Under conventional pose-increment representations, action tokens are sensitive to execution speed and dataset-specific normalization, potentially obscuring geometric structure shared across demonstrations and datasets. We introduce Direction-Scale Decomposition (DSD), an action representation that decomposes translation and rotation increments into direction and scale components before tokenization. DSD isolates motion direction while retaining magnitudes in separate scale channels. We evaluate DSD with uniform binning (BIN) and BEAST, a B-spline-based tokenizer, in simulation and real-world manipulation under both single-dataset and mixed-dataset training. On LIBERO, DSD improves average success rates with both tokenizers. On SimplerEnv, DSD-BIN outperforms BIN by 10.3 percentage points in overall success rate under mixed-dataset training. Real-robot experiments further show gains both with and without robotics pretraining. These results support DSD as an effective action representation for discrete-token VLA models and suggest its potential to mitigate performance degradation when training on large and diverse dataset mixtures. Our project page with additional resources is available at https://vla-dsd.github.io/
Sources
- RT-1: Robotics Transformer for Real-World Control at Scale
- RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policies
- FAST: Efficient Action Tokenization for Vision-Language-Action Models
- $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
- ActionCodec: What Makes for Good Action Tokenizers
- Keypoint Action Tokens Enable In-Context Imitation Learning in Robotics
- SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model
- MoMo: Dial Motion Mode in Robot Manipulation with Spatiotemporal Action Tokenization
- Open X-Embodiment: Robotic Learning Datasets and RT-X Models
- LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning
- Evaluating Real-World Robot Manipulation Policies in Simulation
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models