LAFP: Preserving Latent Action Structure in Latent Policy Learning via Flow Matching
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "LAFP: Preserving Latent Action Structure in Latent Policy Learning via Flow Matching".
Jane: Here's the gist of the paper. Learning high-quality latent actions from large-scale unlabeled videos, coupled with limited real-world interaction data for training an action decoder, has emerged as a promising paradigm for scalable latent policy learning The proposed Latent Action Flow Policy (LAFP) leverages flow matching for latent policy learning and introduces an inference-time interpolation mechanism to mitigate stochasticity-induced misalignment.
Tom: Experimental results demonstrate that LAFP consistently outperforms prior methods on downstream imitation learning tasks, achieving up to 10–15% improvement in success rate while incurring less than 1× additional inference overhead.
Jane: Pruning redundant reasoning tokens The paper introduces Latent Action Flow Policy (LAFP), which addresses the limitations of existing methods by replacing behavior cloning with flow matching for latent policy learning.
Tom: The core issue identified is that directly applying flow matching leads to a misalignment between latent actions and physical actions during action decoder training due to the stochastic nature of the learned policy.
Title and authors: Tom: So we're looking at this paper called "LAFP: Preserving Latent Action Structure in Latent Policy Learning via Flow Matching." Jane, what's the big deal with that title?
Jane: Well, it sounds like they are trying to fix a problem where these latent policies learn actions from videos and then that learning process messes up the underlying structure of what the AI understands about those actions.
Lu: Exactly. They are replacing this old method called behavior cloning with something called flow matching for learning the policy, which they think is better at handling those complex, multi-modal action distributions you see in real life or video data.
Meng: But then they immediately hit a snag: applying flow matching directly to the latent policy causes a mismatch between what the AI *thinks* it should do and what it *actually* does when training the final action decoder.
Tom: It sounds like they found this misalignment is caused by the policy being too random, which is expected with stochastic learning, but they’re proposing a fix for that during the final training stage.
Jane: Right, so instead of just throwing flow matching in there and hoping for the best, they introduced a specific trick to keep things aligned during post-training.
The paper's summary: Tom: So what’s the core idea behind this LAFP thing?
Lu: They set up a whole three-stage process: pre-training, distillation, and then post-training. In the pre-training stage, they train two models simultaneously: an inverse dynamics model and a forward dynamics model under a reconstruction objective to build this latent action space.
Jane: So they aren't just learning actions; they’re trying to learn the rules of motion itself by training these dynamic models together first.
Tom: And then in the distillation stage, instead of using behavior cloning, they use a conditional flow matching policy called pi flow to learn how to map from a simple starting point to the actual latent actions given some input.
Meng: That means they’re using this flow matching process as a way to transport knowledge from that initial simple setup all the way into the more complex, conditional distribution of latent actions.
Lu: And then for post-training, they do something clever: they freeze that learned flow policy and only optimize the action decoder while using this inference-time interpolation mechanism to guide it.
Jane: So if I'm driving a car, instead of just sampling a random action from the learned space, the system is nudged toward a more sensible path during the final training phase.
The paper's improvements: Tom: Okay, so what are they actually claiming is better than what came before?
Lu: The main point is that learning policies through flow matching explicitly preserves the intrinsic structure of the latent action space they built in pre-training. This means it keeps the multi-modal nature of those actions intact, unlike behavior cloning which tends to squash everything into one simple shape.
Jane: That makes sense if you think about it as keeping all the different ways a robot could move—walking, driving, pushing—represented distinctly in that latent space rather than blending them together.
Meng: They show that this preservation of structure leads to consistent improvements on downstream imitation learning tasks when tested on sixteen procedurally generated game-like environments called PROCGEN.
Tom: And the numbers they give are pretty solid: LAFP consistently outperforms another method called LAOM, achieving up to fifteen percent improvement in success rate on those tasks.
Lu: They also found that during inference, they use interpolation-based sampling, which constrains the process by encouraging a latent variable z hat t to be close to the target latent action z t.
Jane: That constraint is what reduces the randomness and keeps things consistent when you actually put it into practice.
Conclusion: Tom: So if we boil this down, LAFP uses flow matching for policy learning and an interpolation mechanism during inference to make sure the actions match up correctly after training.
Jane: It seems like the authors are saying that just switching to flow matching isn't enough; you have to actively manage how those learned actions are sampled during deployment to keep them useful.
Lu: The implication for representation learning is that the latent space itself needs to be structured in a way that supports flow matching, and if you build it right, the policy learning becomes much more stable.
Meng: From a production standpoint, they mention that using only three inference steps provides the best balance between getting good results and keeping the computational cost down.
Lalam: I'm confident in this direction. The evidence on those sixteen PROCGEN tasks shows that preserving structure via flow matching gives you a real edge over just using standard behavior cloning when dealing with diverse actions.
Jiexi Lyu, Xizhou Bu, Qingqiu Huang, Chufeng Tang, Xiaoshuai Hao, Hongbo Wang, Wei Li
Fudan University
cs.CV
Submitted: 2026-06-09
Updated: 2026-11-07
Comments: NeurIPS 2026
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 92/100
The gist: Learning high-quality latent actions from large-scale unlabeled videos, coupled with limited real-world interaction data for training an action decoder…
Key concepts
- Flow Matching
- A technique used for policy learning that models a transport process between a simple starting distribution (prior) and the desired target distribution of latent actions. This method is chosen because it explicitly preserves the geometric structure of the latent action space, which is crucial for accurate control.
- Inference-Time Interpolation
- A mechanism introduced during inference that mitigates misalignment caused by the stochastic nature of flow matching. It works by interpolating between a source Gaussian noise distribution and the target latent action distribution, constraining sampling to encourage better consistency with the underlying latent structure.
- Latent Action Space Structure Preservation
- The goal is to ensure that the learned latent actions maintain their inherent physical or semantic relationships established during pre-training. Flow matching helps achieve this by learning a transport process that respects the geometry of the space, preventing actions from becoming inconsistent with the learned dynamics.
- Action Decoder Constraint
- A regularization technique applied during pre-training where an action decoder constraint is incorporated. This forces the latent action distribution to be more structured and better separated, which stabilizes flow matching and improves downstream imitation learning performance.
Terminology
Summary
Learning high-quality latent actions from large-scale unlabeled videos, coupled with limited real-world interaction data for training an action decoder, has emerged as a promising paradigm for scalable latent policy learning The proposed Latent Action Flow Policy (LAFP) leverages flow matching for latent policy learning and introduces an inference-time interpolation mechanism to mitigate stochasticity-induced misalignment. p. 0 Experimental results demonstrate that LAFP consistently outperforms prior methods on downstream imitation learning tasks, achieving up to 10–15% improvement in success rate while incurring less than 1× additional inference overhead. p. 0
Pruning redundant reasoning tokens
The paper introduces Latent Action Flow Policy (LAFP), which addresses the limitations of existing methods by replacing behavior cloning with flow matching for latent policy learning. The core issue identified is that directly applying flow matching leads to a misalignment between latent actions and physical actions during action decoder training due to the stochastic nature of the learned policy. p. 0 p. 0 To solve this, LAFP utilizes flow matching for latent policy learning while introducing an inference-time interpolation mechanism to mitigate this stochasticity-induced misalignment. p. 0
How the training signal is built
The framework follows a three-stage pipeline: pre-training, distillation, and post-training. In the pre-training stage, a latent action space is established by jointly training an inverse dynamics model (IDM) and a forward dynamics model (FDM) under a reconstruction objective. p. 0 p. 0 The latent space is discretized via a vector-quantized variational autoencoder (VQ-VAE), and the overall model is optimized using the combined objective L = Lvq + Lreconstruction + λ · Lad. p. 2 During distillation, instead of behavior cloning, a conditional flow matching policy πflow learns a transport process from a simple prior to the conditional distribution of latent actions given ot. p. 3
Where the gains come from
The performance gains stem from two main contributions: explicitly preserving the intrinsic structure of the pretrained latent action space through flow matching and introducing an inference-time interpolation mechanism for training the action decoder. The results on 16 Procgen tasks demonstrate that learning latent policies via flow matching can explicitly preserve the intrinsic structure of the pretrained latent action space, leading to consistent improvements in downstream imitation learning performance. p. 1 p. 1 Furthermore, interpolation-based sampling constrains the generation process by encouraging zˆt ≈ zt, thereby reducing latent variability and preserving consistency with the IDM-induced latent structure. p. 7
Evaluation and Trade-offs
The experiments evaluate the method on PROCGEN [10], a benchmark containing 16 procedurally generated game-like environments. The comparison between LAOM and LAFP consistently shows that LAFP achieves superior performance over LAOM in most cases, with particularly notable gains in tasks that require diverse or multimodal behaviors. p. 6 p. 6 Ablation studies indicate that incorporating an action decoder constraint during pre-training consistently improves downstream success rates by producing a more structured and better-separated latent action distribution. p. 7 p. 7 The analysis of the number of flow steps reveals that a small number (3–5) provides the best balance between performance and efficiency, with diminishing returns for larger values. p. 7 p. 7 The study also confirms that reusing the pretrained IDM decoder benefits LAFP but often harms LAOM, highlighting the importance of preserving latent structure for flow-based policies. p. 8
Limitations and future work
While LAFP demonstrates consistent improvements under the current setting, several aspects remain to be further explored. Future work is motivated by the need for distillation or shortcut learning techniques that can compress the multi-step generation process into a single-step or near single-step policy to improve deployment efficiency. p. 8 Extending the framework to continuous control and online learning remains an open direction for future research. p. 8 The results reinforce that downstream policy performance is closely tied to the stability and semantic consistency of the latent representation itself. p. 8
References
[1] Michael Bain and Claude Sammut. A framework for behavioural cloning. p. 9 In Machine intelligence 15, pages 103–129, 1995
[2] Hongzhe Bi, Hengkai Tan, Shenghao Xie, Zeyuan Wang, Shuhe Huang, Haitian Liu, Ruowen Zhao, Yao Feng, Chendong Xiang, Yinze Rong, et al. Motus: A unified latent action world model. p. 9 arXiv preprint arXiv:2512. p. 913030
[3] Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots. p. 9 arXiv preprint arXiv:2503. p. 914734
[4] Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. π0: A vision-language-action flow model for general robot control. p. 9 arXiv preprint arXiv:2410. p. 924164
[5] Qingwen Bu, Jisong Cai, Li Chen, Xiuqi Cui, Yan Ding, Siyuan Feng, Shenyuan Gao, Xindong He, Xuan Hu, Xu Huang, et al. Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems. p. 9 In IROS 2025
[6] Qingwen Bu, Yanting Yang, Jisong Cai, Shenyuan Gao, Guanghui Ren, Maoqing Yao, Ping Luo, and Hongyang Li. Learning to act anywhere with task-centric latent actions. p. 9 In RSS 2025
[7] Xizhou Bu, Jiexi Lyu, Fulei Sun, Ruichen Yang, Zhiqiang Ma, and Wei Li. Laof: Robust latent action learning with optical flow constraints. p. 9 arXiv preprint arXiv:2511. p. 9 p. 916407
[8] Xiaoyu Chen, Hangxing Wei, Pushi Zhang, Chuheng Zhang, Kaixin Wang, Yanjiang Guo, Rushuai Yang, Yucen Wang, Xinquan Xiao, Li Zhao, et al. Villa-x: enhancing latent action modeling in vision-language-action models. p. 9 In ICLR 2026
[9] Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. p. 9 The International Journal of Robotics Research 44(10-11):1684–1704 2025
[10] Karl Cobbe, Chris Hesse, Jacob Hilton, and John Schulman. Leveraging procedural generation to benchmark reinforcement learning. p. 9 In ICML 2020
[11] Youngjoon Jeong, Junha Chun, and Taesup Kim. Learning to act robustly with view-invariant latent actions. p. 9 arXiv preprint arXiv:2601. p. 9 p. 902994
[12] Albina Klepach, Alexander Nikulin, Ilya Zisman, Denis Tarasov, Alexander Derevyagin, Andrei Polubarov, Nikita Lyubaykin, and Vladislav Kurenkov. Object-centric latent action learning. p. 9 In 7th Robot Learning Workshop: Towards Robots with Human-Level Abilities 2025
[13] Tianhong Li and Kaiming He. Back to basics: Let denoising generative models denoise. p. 9 arXiv preprint arXiv:2511.13720
[14] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. p. 1 arXiv preprint arXiv:2210. p. 902747
[15] Mingyu Liu, Jiuhe Shu, Hui Chen, Zeju Li, Canyu Zhao, Jiange Yang, Shenyuan Gao, Hao Chen, and Chunhua Shen. Stamo: Unsupervised learning of generalizable robot motion from compact state representation. p. 9 arXiv preprint arXiv:2510. p. 905057
[16] Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. p. 9 arXiv preprint arXiv:2209. p. 903003
[17] Robert McCarthy, Daniel CH Tan, Dominik Schmidt, Fernando Acero, Nathan Herr, Yilun Du, Thomas G Thuruthel, and Zhibin Li. Towards generalist robot learning from internet video: A survey. p. 9 Journal of Artificial Intelligence Research 83 2025
[18] Dujun Nie, Fengjiao Chen, Qi Lv, Jun Kuang, Xiaoyu Li, Xuezhi Cao, and Xunliang Cai. Lary: A latent action representation yielding benchmark for generalizable vision-to-action alignment. p. 9 arXiv preprint arXiv:2604. p. 9 p. 911689
[19] Alexander Nikulin, Ilya Zisman, Denis Tarasov, Nikita Lyubaykin, Andrei Polubarov, Igor Kiselev, and Vladislav Kurenkov. Latent action learning requires supervision in the presence of distractors. p. 1 In ICML 2025
[20] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. p. 8 arXiv preprint arXiv:2304. p. 907193
[21] Dominik Schmidt and Minqi Jiang. Learning to act without actions. p. 9 In ICLR 2024
[22] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. p. 3 arXiv preprint arXiv:1707. p. 906347
[23] Mustafa Shukor, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zouitine, Michel Aractingi, Caroline Pascal, Martino Russi, Andres Marafioti, et al. Smolvla: A vision-language-action model for affordable and efficient robotics. p. 9 arXiv preprint arXiv:2506. p. 901844
[24] Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. p. 8 In NeurIPS 2017
[25] Junjie Wen, Yichen Zhu, Jinming Li, Minjie Zhu, Zhibin Tang, Kun Wu, Zhiyuan Xu, Ning Liu, Ran Cheng, Chaomin Shen, et al. Tinyvla: Towards fast, data-efficient vision-language-action models for robotic manipulation. p. 0 IEEE Robotics and Automation Letters 2025
[26] Chengen Xie, Bin Sun, Tianyu Li, Junjie Wu, Zhihui Hao, XianPeng Lang, and Hongyang Li. Latentvla: Efficient vision-language models for autonomous driving via latent action prediction. p. 0 arXiv preprint arXiv:2601.05611
[27] Ge Yan, Jiyue Zhu, Yuquan Deng, Shiqi Yang, Ri-Zhao Qiu, Xuxin Cheng, Marius Memmel, Ranjay Krishna, Ankit Goyal, Xiaolong Wang, et al. Maniflow: A general robot manipulation policy via consistency flow training. p. 1 arXiv preprint arXiv:2509. p. 901819
[28] Jiange Yang, Yansong Shi, Haoyi Zhu, Mingyu Liu, Kaijing Ma, Yating Wang, Gangshan Wu, Tong He, and Limin Wang. Como: Learning continuous latent motion from internet videos for scalable robot learning. p. 0 arXiv preprint arXiv:2505. p. 917006
[29] Seonghyeon Ye, Joel Jang, Byeongguk Jeon, Sejune Joo, Jianwei Yang, Baolin Peng, Ajay Mandlekar, Reuben Tan, Yu-Wei Chao, Bill Yuchen Lin, et al. Latent action pretraining from videos. p. 0 In ICLR 2025
[30] Xinlei Yu, Zhangquan Chen, Yongbo He, Tianyu Fu, Cheng Yang, Chengming Xu, Yue Ma, Xiaobin Hu, Zhe Cao, Jie Xu, et al. The latent space: Foundation evolution mechanism ability and outlook. p. 1 arXiv preprint arXiv:2604.02029
[31] Chuheng Zhang, Tim Pearce, Pushi Zhang, Kaixin Wang, Xiaoyu Chen, Wei Shen, Li Zhao, and Jiang Bian. What do latent action models actually learn? p. 1 In NeurIPS 2025
--- Page 1 ---
LAFP: Preserving Latent Action Structure in Latent Policy Learning via Flow Matching
Abstract
Learning high-quality latent actions from large-scale unlabeled videos, coupled with limited real-world interaction data for training an action decoder has emerged as a promising paradigm for scalable latent policy learning. However, existing approaches typically rely on behavior cloning, which tends to collapse inherently multimodal action distributions into unimodal ones, thereby degrading the pretrained latent action structure. p. 0 While flow matching provides a potential alternative, directly applying it leads to a misalignment between latent actions and physical actions during action decoder training, due to the stochastic nature of the learned policy. p. 0 To address this, LAFP proposes Latent Action Flow Policy, which leverages flow matching for latent policy learning and introduces an inference-time interpolation mechanism to mitigate stochasticity-induced misalignment. p. 0
The paper introduces Latent Action Flow Policy (LAFP), which addresses the limitations of existing methods by replacing behavior cloning with flow matching for latent policy learning. The core issue identified is that directly applying flow matching leads to a misalignment between latent actions and physical actions during action decoder training due to the stochastic nature of the learned policy.
The framework follows a three-stage pipeline: pre-training, distillation, and post-training. In the pre-training stage, a latent action space is established by jointly training an inverse dynamics model (IDM) and a forward dynamics model (FDM) under a reconstruction objective. The distillation stage employs a conditional flow matching policy πflow that learns a transport process from a simple prior to the conditional distribution of latent actions given ot. p. 3
The performance gains stem from two main contributions: explicitly preserving the intrinsic structure of the pretrained latent action space through flow matching and introducing an inference-time interpolation mechanism for training the action decoder. The results on 16 Procgen tasks demonstrate that learning latent policies via flow matching can explicitly preserve the intrinsic structure of the pretrained latent action space, leading to consistent improvements in downstream imitation learning performance.
The experiments evaluate the method on PROCGEN [10], a benchmark containing 16 procedurally generated game-like environments. The comparison between LAOM and LAFP consistently shows that LAFP achieves superior performance over LAOM in most cases, with particularly notable gains in tasks that require diverse or multimodal behaviors. Ablation studies indicate that incorporating an action decoder constraint during pre-training consistently improves downstream success rates by producing a more structured and better-separated latent action distribution. The analysis of the number of flow steps reveals that a small number (3–5) provides the best balance between performance and efficiency, with diminishing returns for larger values.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Preserve latent action structure during policy optimization by leveraging flow matching for latent policy learning, which
explicitly preserve[s] the intrinsic structure of the pretrained latent action space.
This leads to aconsistent improvement in downstream imitation learning performance
and better capturesmultimodal action distributions.
-
Introduce an inference-time interpolation mechanism to mitigate stochasticity during post-training, specifically by interpolating between the source Gaussian noise distribution and the target latent action distribution. This resolves the problem where
sampling directly from noise yields multiple plausible latent zt, causing misalignment with action labels.
-
Achieve performance comparable to an action decoder directly trained with IDM supervision by using flow matching inference. The system can achieve this by employing
interpolation-based sampling
during inference, whichconstrains the source distribution in flow matching to interpolate between Gaussian noise and zt, reducing stochasticity and encouraging zt ≈ zt.
-
Enhance downstream task performance on imitation learning tasks by replacing behavior cloning with flow matching. The paper shows that LAFP consistently
outperforms prior methods on downstream imitation learning tasks, achieving up to 10–15% improvement in success rate.
-
Improve the stability and robustness of the latent-to-action correspondence by enforcing action decoder regularization during pre-training. This is because
incorporating this regularization consistently improves downstream success rates by producing a more structured and better-separated latent action distribution,
which isespecially important for flow matching policy, which rely on accurately modeling the geometry of the latent space.
-
Optimize computational efficiency in deployment by using a small number of flow steps during inference. The study shows that
using only three inference steps provides the best balance between performance and efficiency,
demonstrating that flow matching can achieve substantial improvementswith only modest additional computational cost.
Sources
- Motus: A Unified Latent Action World Model
- GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- LAOF: Robust Latent Action Learning with Optical Flow Constraints
- Learning to Act Robustly with View-Invariant Latent Actions
- Back to Basics: Let Denoising Generative Models Denoise
- Flow Matching for Generative Modeling
- StaMo: Unsupervised Learning of Generalizable Robot Motion from Compact State Representation
- Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
- LARY: A Latent Action Representation Yielding Benchmark for Generalizable Vision-to-Action Alignment
- DINOv2: Learning Robust Visual Features without Supervision
- Proximal Policy Optimization Algorithms
- SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics
- FLARE: Learning Future-Aware Latent Representations from Vision-Language Models for Autonomous Driving
- ManiFlow: A General Robot Manipulation Policy via Consistency Flow Training
- CoMo: Learning Continuous Latent Motion from Internet Videos for Scalable Robot Learning
- The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs