StageVLN: Spatial and Trajectory Auxiliary Guidance for Efficient Vision-Language Navigation
cs.CV, cs.RO
Submitted: 2026-10-05
Updated: 2026-10-05
License: http://creativecommons.org/licenses/by/4.0/
The gist: Vision-and-Language Navigation (VLN) policies increasingly benefit from strong semantic priors provided by large vision-language models (VLMs).
Abstract
Vision-and-Language Navigation (VLN) policies increasingly benefit from strong semantic priors provided by large vision-language models (VLMs). However, standard action supervision does not explicitly encourage intermediate representations to preserve scene geometry, relative orientation, or global episode progress. Incorporating depth estimators, explicit maps, point clouds, or geometry foundation models at inference can provide such structure but introduces additional computation, memory overhead, and architectural dependence during deployment. We introduce StageVLN, a training framework that shapes navigation representations through privileged spatial and trajectory guidance while preserving the original inference pathway. A frozen geometry foundation model provides multi-level spatial guidance to hierarchical navigator states, while relative-heading and expert-route progress objectives provide complementary trajectory-state supervision. All auxiliary components are used only during training and removed at deployment. On R2R-CE validation-unseen, StageVLN achieves 56.3% SR and 51.4% SPL with a 4B-parameter backbone, without an additional geometry encoder at inference. On RxR-CE, it achieves 54.3% SR without additional navigation training data or a geometry encoder at inference.
Sources
- NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation
- Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Tasks
- NaVILA: Legged Robot Vision-Language-Action Model for Navigation
- StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modeling
- VGGT-$\Omega$
- SpatialStack: Layered Geometry-Language Fusion for 3D VLM Spatial Reasoning
- Learning Geometric Representations from Videos for Spatial Intelligent Multimodal Large Language Models
- Distilling the Knowledge in a Neural Network
- Unifying distillation and privileged information
- FitNets: Hints for Thin Deep Nets
- Self-Monitoring Navigation Agent via Auxiliary Progress Estimation
- Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Tasks
- A Recurrent Vision-and-Language BERT for Navigation
- 1st Place Solutions for RxR-Habitat Vision-and-Language Navigation Competition (CVPR 2022)
- InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environment
- Sim-to-Real Transfer via 3D Feature Fields for Vision-and-Language Navigation
- ActiveVLN: Towards Active Exploration via Multi-Turn RL in Vision-and-Language Navigation
- Matterport3D: Learning from RGB-D Data in Indoor Environments
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models