Q-VGM: Q-Guided Value-Gradient Matching for Offline-to-Online RL of Flow-Matching VLA Policies
cs.RO
Submitted: 2026-06-06
Updated: 2026-09-17
Comments: Revised manuscript. 8 pages, 4 figures, 6 tables
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
- Proximal Policy Optimization Algorithms
- $\pi_\texttt{RL}$: Online RL Fine-tuning for Flow-based Vision-Language-Action Models
- Flow Matching Policy Gradients
- Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning
- $\pi^{*}_{0.6}$: a VLA That Learns From Experience
- RL Token: Bootstrapping Online RL with Vision-Language-Action Models
- Hume: Introducing System-2 Thinking in Visual-Language-Action Model
- Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Control
- Fine-tuning of diffusion models via stochastic control: entropy regularization and beyond
- Trust Region Q Adjoint Matching
- Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success
- FAST: Efficient Action Tokenization for Vision-Language-Action Models
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies