Mixture of Frames Policy: Multi-Frame Action Denoising for Bimanual Mobile Manipulation
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: We're looking at the paper: "Mixture of Frames Policy".
Dev: Here's the gist of the paper. Mixture of Frames Policy (MoF) is a diffusion policy that performs synchronized action denoising across multiple coordinate frames to improve performance in bimanual mobile manipulation tasks The best action frame is task-dependent and that MoF improves over oracle frame selection and standard Mixture-of-Experts (MoE) baselines Problem Statement Robotic manipulation systems are inherently multi-frame, as local actions may be simple in an end-effector frame, while transport, upright-object handling, and whole-body coordination are better represented in a base-aligned frame Modern visuomotor policies typically commit to a single predefined action frame, forcing one denoiser to model action distributions that are often unnecessarily complex in that frame This suggests that frames should not only be provided as observations, but should also define alternative action spaces in which denoising can be performed Consistent Multi-Frame Denoising MoF maintains a single canonical diffusion state, re-expresses it in several task-relevant frames, applies frame-specialized denoisers, and fuses their noise predictions back in the canonical frame Specifically, let Fc denote the canonical frame in which the scheduler state x k c is stored For each frame Fm ∈ F, we transform the canonical-frame noisy action sample x k c into the expert frame via mTc(·), feed it to a frame-specialized denoiser ϵ θ m, transform the predicted noise back to the canonical frame using cTm(·), and fuse the aligned predictions This synchronized update has two benefits: First, all experts contribute to the same underlying diffusion trajectory, avoiding the mode-mismatch problem Second, because fusion occurs in the canonical noise space, the diffusion scheduler remains unchanged Transformation-Compatible Action Representation The synchronized denoising procedure in Sec.
Rosa: 3.2 requires frame transformations to be applied directly to intermediate diffusion states, whose rotation channels generally do not form a valid element of SO(3) This poses a challenge for the row-vector 6D rotation representation used in standard Diffusion Policy implementations, which stores the first two rows of the rotation matrix We instead use a column-vector rotation representation For one robot arm at one action timestep, the pose/action channels expressed in the canonical frame Fc are xc =
pc, c 1 c, c 2 c, g: , where pc is the end-effector position This operation is well-defined even when c 1 c and c 2 c are arbitrary noisy vectors rather than columns of a valid rotation matrix Where the gains come from The within-episode router dynamics track the visible phase structure of each task, suggesting that MoF-MoE’s advantage over Oracle Frame stems primarily from its ability to assign different frames to different task phases The two tasks with the largest router modulation, Threading and Drawer Cleanup, are also the tasks where MoF-MoE most improves over Oracle Frame This relationship suggests that MoF-MoE’s advantage over the best single frame stems primarily from its ability to assign different frames to different task phases, rather than from a static preference for any one frame Pruning redundant reasoning tokens The orthogonalization ablation validates the transformation-compatible action representation When noisy rotation channels are projected back to SO(3) before applying frame transformations, MoFMoE drops by 7.0 percentage points on average, and MoF-Ensemble collapses completely This is because the orthogonalization completely destroys all non-canonical-frame experts Removing the best-performing single frame from the expert set reduces performance by 6.2 percentage points, showing that the expert set is not redundant and that task-relevant frames contribute complementary denoising structure How the training signal is built The mixed objective Lmix supervises only the fused canonical-frame prediction, so experts with small router weights may receive weak gradients To stabilize training, we add a per-expert auxiliary denoising loss that directly supervises each expert in its own frame The final training objective is L = Lmix +λauxLaux This suggests that directly supervising each expert in its own frame is important for keeping all frame experts meaningful during training We present Mixture of Frames Policy (MoF), a diffusion-policy framework for denoising actions across multiple coordinate-frame parameterizations Our experiments show that action-frame choice has a substantial and task-dependent effect on diffusion-policy performance, and that fusing framespecialized denoisers improves over baselines including oracle task-level frame selection MoF also matches or outperforms Oracle Frame on most tasks The largest improvement is on Threading (+7.7 points) In the pouring task, MoF-MoE achieves 85% success, with only three failures in picking up (1 left, 2 right) The Left policy achieves only 15% success, with 12 right-cup pickup failures, two left-cup pickup failures, two pouring failures, and one rollout that repeatedly continues the pouring motion even after successful transfer The Right policy achieves 70% success, but all of its failures are left-side failures The Rel Traj policy achieves only 5% success and is highly unstable during precise pickup The Left policy achieves 55% success and fails nine times at cup pickup The Right policy fails in all 20 rollouts and exhibits highly unstable navigation throughout Limitations First, while theoretically adaptable to larger vision-language action (VLA) models, our current experiments focus on smaller diffusion policies Second, the router is supervised solely via denoising loss, which might not strictly correspond to policy performance Third, MoF relies on a designer-specified set of candidate frames and accurate frame transformations from robot proprioception Future work could investigate learning or automatically discovering task-relevant frames and study how frame uncertainty affects multi-frame denoising in real-world deployment Acknowledgments The authors would like to thank Yifan Hou for the help during RB-Y1 controller implementation, and Mengda Xu for the helpful discussion during project brainstorming This work was supported in part by the NSF Award and Apple, and Toyota Research Institute The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of the sponsors References
1: C.
Dev: Chi, Z.
Rosa: Xu, S.
Title and authors: Rosa: So what's the actual mechanism they propose? They describe it as maintaining a single canonical diffusion state, but then re-expressing that state into several task-relevant frames and feeding those into frame-specialized denoisers.
Dev: They do this by taking the canonical frame, say Fc, and transforming it to an expert frame Fm using a transformation mTc(·), running the expert denoiser epsilon theta m m, and then transforming that noise prediction back to the canonical frame using cTm(·).
Rosa: That sounds complicated for real-time control. How do you make sure all these experts are actually contributing to the same thing?
Dev: The point is that this synchronized update has two main benefits: first, all experts contribute to the same underlying diffusion trajectory, which avoids that mode mismatch problem we often see in standard Mixture-of-Experts or MoE baselines. Second, because the fusion happens back in the canonical noise space, the diffusion scheduler itself doesn't have to change.
Rosa: That makes sense for stability. So they’re trying to keep the overall process consistent even while switching between frame perspectives?
Dev: Yeah, they are trying to make sure that when you fuse those predictions, you're fusing them in a way that respects the diffusion process itself. But there’s a practical hurdle they run into with how they represent rotation channels.
The paper's summary: Rosa: They hit this problem where the transformation-compatible action representation is tricky because the rotation channels don't always form a valid element of SO(three), which clashes with how standard Diffusion Policy implementations handle their 6D rotation representations, like using row vectors <ref:2607.11884#pg3>.
Dev: So they switch things up and use a column-vector representation instead, which makes the frame transformations work even when the input vectors for c1 c and c2 c are just arbitrary noisy vectors instead of guaranteed columns of a valid rotation matrix.
Rosa: That’s clever, adapting the math to fit the physics of what they're trying to model. But where do you see this actually helping a robot on the ground?
Dev: The paper suggests that the gains come from how within-episode router dynamics track visible phase structure of each task. They found that MoF outperforms oracle frame selection and standard Mixture-of-Experts baselines because it can assign different frames to different task phases.
Rosa: So, it's not about picking one perfect frame for the whole episode; it's about letting the system decide which frame to use based on what the robot is currently doing?
Dev: Precisely. They point out that the two tasks with the biggest router modulation, Threading and Drawer Cleanup, are where MoF-MoE gets its most improvement over Oracle Frame selection. This suggests it's about adapting to phase changes rather than just sticking to one static frame.
The paper's improvements: Rosa: Beyond just the frame switching, they also talk about how they make the training signal better. They use a mixed objective Lmix that only supervises the fused canonical-frame prediction, which can leave experts with small router weights getting weak gradients.
Dev: To stabilize that, they add a per-expert auxiliary denoising loss that directly supervises each expert in its own frame. That way, even if an expert isn't heavily used by the router, it still gets a direct signal about its performance in its specific frame.
Rosa: So they’re essentially giving each frame expert a personal training objective so they don't just drift away when the main objective is focused on the fused result?
Dev: That’s right. And looking at the pruning ablation, when you project noisy rotation channels back to SO(three) before applying frame transformations, MoF-MoE drops by seven point zero percentage points on average, and MoF-Ensemble collapses completely because those non-canonical experts are just noise <ref:2607.11884#pg3>.
Rosa: That suggests that the expert set isn't redundant; each task-relevant frame actually contributes something complementary to the overall denoising structure.
Conclusion: Dev: So wrapping up, the authors of "Mixture of Frames Policy: Multi-Frame Action Denoising for Bimanual Mobile Manipulation" show that action-frame choice has a substantial effect on performance and that it’s task dependent. They confirm that fusing frame-specialized denoisers improves over baselines including oracle frame selection and standard Mixture-of-Experts (MoE) baselines.
Rosa: The biggest win they highlight is on the Threading task, where MoF-MoE showed a gain of seven point seven points compared to the best single frame approach <ref:2607.11884#pg3>. They also show it matches or outperforms Oracle Frame on most tasks overall, though that improvement is most significant when the task changes its requirements.
Dev: And looking at their results on the pouring task, MoF-MoE achieved eighty-five percent success with only three failures in picking up objects—one left, two right. That’s a big jump compared to the other policies they tested.
Taro: From my testing standpoint, it’s interesting that the router dynamics track visible phase structure of each task; it means MoF-MoE's advantage over Oracle Frame stems primarily from its ability to assign different frames to different task phases, not just a static preference for any one frame.
Rosa: So if you only listen to the show, what does this mean for us? It means that for complex tasks like bimanual manipulation, we shouldn't just pick one fixed way of looking at the action; we need a system that can dynamically switch between frames as the task progresses.
Dev: That’s right. We’ve got a framework here, the Mixture of Frames Policy, which lets us leverage different coordinate systems to model complex movements more effectively in diffusion policies. We'll keep an eye on how they handle real-world deployment and frame uncertainty next.
Dian Wang, Jisang Park, Xiaomeng Xu, Han Zhang, Shuran Song
Stanford University
cs.RO
Submitted: 2026-07-13
Updated: 2026-10-28
Project page: https://mofpo.github.io
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Importance score: 93/100
The gist: Mixture of Frames Policy (MoF) is a diffusion policy that performs synchronized action denoising across multiple coordinate frames to improve performance in bimanual mobile manipulation tasks The…
Key concepts
- Consistent Multi-Frame Denoising
- This method keeps a single main state but transforms it into several task-relevant frames. Specialized denoisers process each frame's view, and their noise predictions are fused back into the main frame's noise space. This ensures all experts work on the same underlying trajectory, avoiding confusion between different views.
- Transformation-Compatible Action Representation
- The paper solves a problem where rotating action vectors don't fit standard 6D rotation formats. They use a column-vector representation for actions in the canonical frame, which allows them to apply necessary frame transformations directly to intermediate diffusion states without breaking the mathematical structure.
- Task-Dependent Frame Selection
- MoF leverages the idea that different frames are best for different parts of a task. The system learns to assign specific frames to specific phases of manipulation, such as using one frame for end-effector movement and another for whole-body coordination, maximizing performance gains.
- Per-Expert Auxiliary Denoising Loss
- During training, the main loss only focuses on the final fused prediction. To keep all frame experts useful, a secondary loss is added that forces each expert to accurately predict noise in its own specific frame. This prevents less important experts from becoming inactive during learning.
Terminology
Summary
Mixture of Frames Policy (MoF) is a diffusion policy that performs synchronized action denoising across multiple coordinate frames to improve performance in bimanual mobile manipulation tasks The best action frame is task-dependent and that MoF improves over oracle frame selection and standard Mixture-of-Experts (MoE) baselines
Problem Statement
Robotic manipulation systems are inherently multi-frame, as local actions may be simple in an end-effector frame, while transport, upright-object handling, and whole-body coordination are better represented in a base-aligned frame Modern visuomotor policies typically commit to a single predefined action frame, forcing one denoiser to model action distributions that are often unnecessarily complex in that frame This suggests that frames should not only be provided as observations, but should also define alternative action spaces in which denoising can be performed
Consistent Multi-Frame Denoising
MoF maintains a single canonical diffusion state, re-expresses it in several task-relevant frames, applies frame-specialized denoisers, and fuses their noise predictions back in the canonical frame Specifically, let Fc denote the canonical frame in which the scheduler state x k c is stored For each frame Fm ∈ F, we transform the canonical-frame noisy action sample x k c into the expert frame via mTc(·), feed it to a frame-specialized denoiser ϵ θ m, transform the predicted noise back to the canonical frame using cTm(·), and fuse the aligned predictions This synchronized update has two benefits: First, all experts contribute to the same underlying diffusion trajectory, avoiding the mode-mismatch problem Second, because fusion occurs in the canonical noise space, the diffusion scheduler remains unchanged
Transformation-Compatible Action Representation
The synchronized denoising procedure in Sec. 3.2 requires frame transformations to be applied directly to intermediate diffusion states, whose rotation channels generally do not form a valid element of SO(3) This poses a challenge for the row-vector 6D rotation representation used in standard Diffusion Policy implementations, which stores the first two rows of the rotation matrix We instead use a column-vector rotation representation For one robot arm at one action timestep, the pose/action channels expressed in the canonical frame Fc are xc = [pc, c 1 c, c 2 c, g], where pc is the end-effector position This operation is well-defined even when c 1 c and c 2 c are arbitrary noisy vectors rather than columns of a valid rotation matrix
Where the gains come from
The within-episode router dynamics track the visible phase structure of each task, suggesting that MoF-MoE’s advantage over Oracle Frame stems primarily from its ability to assign different frames to different task phases The two tasks with the largest router modulation, Threading and Drawer Cleanup, are also the tasks where MoF-MoE most improves over Oracle Frame This relationship suggests that MoF-MoE’s advantage over the best single frame stems primarily from its ability to assign different frames to different task phases, rather than from a static preference for any one frame
Pruning redundant reasoning tokens
The orthogonalization ablation validates the transformation-compatible action representation When noisy rotation channels are projected back to SO(3) before applying frame transformations, MoFMoE drops by 7.0 percentage points on average, and MoF-Ensemble collapses completely This is because the orthogonalization completely destroys all non-canonical-frame experts Removing the best-performing single frame from the expert set reduces performance by 6. p. 6 5 points,
suggesting that MoF is robust to the arbitrary choice of canonical frame. In contrast, removing the best-performing single frame from the expert
set reduces performance by 6.Where the paper says it
How the training signal is built
The mixed objective Lmix supervises only the fused canonical-frame prediction, so experts with small router weights may receive weak gradients To stabilize training, we add a per-expert auxiliary denoising loss that directly supervises each expert in its own frame The final training objective is L = Lmix +λauxLaux This suggests that directly supervising each expert in its own frame is important for keeping all frame experts meaningful during training
Conclusion
We present Mixture of Frames Policy (MoF), a diffusion-policy framework for denoising actions across multiple coordinate-frame parameterizations Our experiments show that action-frame choice has a substantial and task-dependent effect on diffusion-policy performance, and that fusing framespecialized denoisers improves over baselines including oracle task-level frame selection MoF also matches or outperforms Oracle Frame on most tasks The largest improvement is on Threading (+7.7 points) In the pouring task, MoF-MoE achieves 85% success, with only three failures in picking up (1 left, 2 right) The Left policy achieves only 15% success, with 12 right-cup pickup failures, two left-cup pickup failures, two pouring failures, and one rollout that repeatedly continues the pouring motion even after successful transfer The Right policy achieves 70% success, but all of its failures are left-side failures The Rel Traj policy achieves only 5% success and is highly unstable during precise pickup The Left policy achieves 55% success and fails nine times at cup pickup The Right policy fails in all 20 rollouts and exhibits highly unstable navigation throughout
Limitations
First, while theoretically adaptable to larger vision-language action (VLA) models, our current experiments focus on smaller diffusion policies Second, the router is supervised solely via denoising loss, which might not strictly correspond to policy performance Third, MoF relies on a designer-specified set of candidate frames and accurate frame transformations from robot proprioception Future work could investigate learning or automatically discovering task-relevant frames and study how frame uncertainty affects multi-frame denoising in real-world deployment
Acknowledgments
The authors would like to thank Yifan Hou for the help during RB-Y1 controller implementation, and Mengda Xu for the helpful discussion during project brainstorming This work was supported in part by the NSF Award and Apple, and Toyota Research Institute The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of the sponsors
References
[1] C. Chi, Z. Xu, S. Feng, E. Cousineau, Y. p. 8 References
[1] C. Cousineau, Y. Cousineau, Y. Burchfiel, R. Burchfiel, R. Tedrake, and S.Where the paper says it
Where the paper says it
Where the paper says it
[2] M. Liu, X. Li, Z. Ling, Y. Li, and H. Su
[3] M. R. Montero, G. p. 8 PMLR,
2023. Montero, G. Montero, G. Franzese, J. Franzese, J. Kober, and C.Where the paper says it
Where the paper says it
Where the paper says it
[4] J. Grannen, Y. p. 8 IEEE,
2024. Grannen, Y.Where the paper says it
[5] A. C.-W. Lee, I. Chuang, L.-Y Chen, and I Soltani
[6] J.-J Jiang, X.-M Wu, Y.-X He, L.-A Zeng, Y.-L Wei, D Zhang, and W-S Zheng
[7] Z. Chen, N. T Chan, Y Hou, C Tie, Z Liu, H Chen, J Chen, J Shi et al
[8] J Yang, I Huang, B Vu, M Bajracharya R Antonova and J Bohg
[9] H Ha, Y Gao, Z Fu, J Tan, and S Song
[10] P Sundaresan, R Malhotra, P Miao, J Yang et al
[11] Z Fu, T Z Zhao, and C Finn
[12] L Y Zhu, P Kuppili, R Punamiya et al
[13] Y Jiang, R Zhang, J Wong, C Wang et al
[14] X Xu, J Park, H Zhang, E Cousineau A Bhat et al
[15] L Righetti, M Kalakrishnan, P Pastor et al
[16] Y Huang, J Silvério, L Rozo, and D Caldwell
[17] H Zhao, D Wang, Y Zhu, X Zhu et al
[18] H Zhao, Y Qi, B Hu, Y Zhu et al
[19] J Gao, X Jin, F Krebs, N Jaquier and T Asfour
[20] A Bahety, P Mandikal, B Abbatematteo and R Martín-Martín
[21] R A Jacobs, M I Jordan, S J Nowlan and G E Hinton
[22] N Shazeer, A Mirhoseini, K Maziarz et al
[23] O Celik, D Zhou, G Li, P Becker and G Neumann
[24] O Celik, A Taranovic, and G Neumann
[25] Y Wang, Y Zhang, M Huo, T Tian et al
[26] K Guo, H Liu, Y Sun, R Zhao J Zhou and J Ma
[27] C Hao, X Zhai, Y Liu, and H Soh
[28] A Rodriguez, C Li, L Mazza, R Younis et al
[29] B Cheng, T Liang, S Huang, M Shao et al
[30] M Reuss, J Pari, P Agrawal and R Lioutikov
[31] H Zhou, D Blessing, G Li, O Celik X Jia et al
[32] J Yu, H Liu, Q Yu, J Ren C Hao et al
[33] D Wang, S Hart, D Surovik T Kelestemur H Huang et al
[34] X Xu, D Bauer, and S Song
[35] D Wang, B Hu, S Song R Walters and R Platt
[36] X Xu, Y Hou, Z Liu, and S Song
[37] N Chernyadev, N Backshall, X Ma, Y Lu et al
[38] Z Jiang, Y Xie, K Lin, Z Xu W Wan A Mandlekar L J Fan and Y Zhu
[39] J Song, C Meng, and S Ermon
(Note: The paper provided only goes up to Page 20, so page numbers in the references will be based on the structure provided.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Improved action representation allows for exact, differentiable frame transformations without requiring noisy rotations to lie on the SO(3) manifold, which enables consistent multi-frame denoising across different coordinate systems during action generation.
-
The Mixture of Frames Policy (MoF) can reason over multiple coordinate frames simultaneously by instantiating
one expert denoiser per reference frame and fusing their predictions at each denoising step through either a mixture-of-experts pipeline or simple ensembling.
-
The system can adapt its action representation based on the task phase, as seen in the router analysis where
the rel-traj expert is the dominant frame during the first part of the episode, where each gripper reaches and grasps its object, and the base expert takes over for the later contact-rich phase.
-
The policy can achieve superior performance by leveraging task-dependent frame choice, as demonstrated by MoF improving over
oracle frame selection and standard Mixture-of-Experts (MoE) baselines,
leading to gains like+7.7 points
on the Threading task. -
The system can be made more robust to arbitrary canonical frame choices because ablation studies show that
Changing the canonical frame has only a modest effect,
indicating that MoF isrobust to the arbitrary choice of canonical frame.
-
The system can be trained more effectively by incorporating per-expert auxiliary denoising loss, which supervises each expert directly against the diffusion noise expressed in its own frame, preventing experts with small router weights from drifting away.
Abstract
Robotic manipulation is inherently multi-frame: local actions may be simple in an end-effector frame, while transport, upright-object handling, and whole-body coordination are better represented in a base-aligned frame. However, modern diffusion-based visuomotor policies typically commit to a single predefined action frame, forcing one denoiser to model action distributions that are often unnecessarily complex in that frame. We propose Mixture of Frames Policy (MoF), a diffusion policy that performs synchronized action denoising across multiple coordinate frames. MoF maintains a single canonical diffusion state, re-expresses it in several task-relevant frames, applies frame-specialized denoisers, and fuses their noise predictions back in the canonical frame. To make this possible for intermediate noisy diffusion states, we adopt a column-based 6D rotation representation within an SE(3) action parameterization that supports exact, differentiable frame transformations without requiring noisy rotations to lie on the SO(3) manifold. Across nine simulated bimanual manipulation tasks, we show that the best action frame is task-dependent and that MoF improves over oracle frame selection and standard Mixture-of-Experts (MoE) baselines. We further evaluate MoF on two real-world bimanual mobile manipulation tasks, demonstrating that it outperforms all constituent single-frame baselines.
Sources
- RoTri-Diff: A Spatial Robot-Object Triadic Interaction-Guided Diffusion Model for Bimanual Manipulation
- Mobi-$\pi$: Mobilizing Your Robot Learning Policy
- HoMeR: Learning In-the-Wild Mobile Manipulation via Hybrid Imitation and Whole-Body Control
- HoMMI: Learning Whole-Body Mobile Manipulation from Human Demonstrations
- MoE-ACT: Scaling Multi-Task Bimanual Manipulation with Sparse Task-Conditioned Mixture-of-Experts Transformers
- Abstracting Robot Manipulation Skills via Mixture-of-Experts Diffusion Policies
- LAR-MoE: Latent-Aligned Routing for Mixture of Experts in Robotic Imitation Learning
- MoE-DP: An MoE-Enhanced Diffusion Policy for Robust Long-Horizon Robotic Manipulation with Skill Decomposition and Failure Recovery
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation