From Recovery to Drop-off: How Action Post-training Reduces a VLM's Late-Layer Depth Decodability

arXiv:2608.08904 · cs.CV, cs.AI, cs.LG · Submitted 2026-08-14 · Read on arXiv

Alexander Hackett, Arnaud Denis-Remillard, Axel Cassou

cs.CV, cs.AI, cs.LG

Submitted: 2026-08-14

Updated: 2026-08-17

Comments: Accepted to the archival proceedings track of the Embodied Multimodal Reasoning (EMR) Workshop at ECCV 2026

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Decoder — 20× in this paper · explained in 2 episodes
• Transformer — 15× in this paper · explained in 2 episodes
• Ablation — 14× in this paper · explained in 1 episode
• Localization — 7× in this paper · explained in 2 episodes
• Decode — 5× in this paper · explained in 2 episodes
• MLLMs — 4× in this paper · explained in 2 episodes
• Dense Prediction Transformer — 3× in this paper
• Foundation Model — 2× in this paper · explained in 3 episodes
• Foundation Models — 2× in this paper · explained in 8 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Action Flow — 1× in this paper · explained in 1 episode
• Benchmarking — 1× in this paper · explained in 4 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Flow Matching — 1× in this paper · explained in 22 episodes
• Generalization — 1× in this paper · explained in 14 episodes
• Inductive Bias — 1× in this paper · explained in 3 episodes
• Interpretability — 1× in this paper · explained in 15 episodes
• Linear Probe — 1× in this paper · explained in 3 episodes
• Regularization — 1× in this paper · explained in 3 episodes
• Semantic Segmentation — 1× in this paper · explained in 3 episodes
• Semantic Understanding — 1× in this paper · explained in 3 episodes
• Tokenization — 1× in this paper · explained in 2 episodes
• Transformers — 1× in this paper · explained in 2 episodes
• Vision-Language-Action (VLA) Models — 1× in this paper · explained in 5 episodes
• Vision-language models — 1× in this paper · explained in 2 episodes

Sources

• Probing the 3D Awareness of Visual Foundation Models
• $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
• RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
• RT-1: Robotics Transformer for Real-World Control at Scale
• Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better
• MolmoAct2: Action Reasoning Models for Real-world Deployment
• Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space
• Transformer Feed-Forward Layers Are Key-Value Memories
• Don't Blind Your VLA: Aligning Visual Representations for OOD Generalization
• OpenVLA: An Open-Source Vision-Language-Action Model
• Locating and Editing Factual Associations in GPT
• Octo: An Open-Source Generalist Robot Policy
• FAST: Efficient Action Tokenization for Vision-Language-Action Models
• Vision Transformers for Dense Prediction
• SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features
• From Drop-off to Recovery: A Mechanistic Analysis of Segmentation in MLLMs

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models