LMGN: A Unified Policy Learning Model for Last-Mile Grasp-Navigation

arXiv:2611.07515 · cs.RO · Submitted 2026-11-05 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "LMGN: A Unified Policy Learning Model for Last-Mile Grasp-Navigation".

Dev: Here's the gist of the paper. Legged robots equipped with manipulators have shown significant potential in the fields of daily life and industry in complex environments <ref:2611.07515#pg2>.

Rosa: To address these challenges, we propose LastMile Grasp-Navigation (LMGN), a unified policy learning model for Quadruped robot loco-manipulation <ref:2611.07515#pg5>.

Dev: Extensive experiments on the Habitat and Isaac Gym simulators demonstrate that LMGN consistently outperforms state-of-the-art methods, denoting that integrating grasp-aware reachability with stage-aware policy learning significantly improves the success rates of last-mile navigation and target grasp <ref:2611.07515#pg4>.

Rosa: Core Innovation and Representation An Explicit Distance and Directional Reachability (ED-DR) representation is first introduced to explicitly represent the features of grasp-compatible base placement and approach direction <ref:2611.07515#pg3>.

Paper summary: Dev: So wrapping up this look at "LMGN: A Unified Policy Learning Model for Last-Mile Grasp-Navigation" by Hu and the team, the core idea is using an Explicit Distance and Directional Reachability representation to explicitly represent base placement and approach direction features > They build a Hierarchical MixtureofExperts policy to model those distinct behaviors across the stages of movement.

Rosa: The authors claim that their proposed joint optimization method aligns navigation, expert routing, and action-level consistency by integrating the RL loss, the navigation loss, and the grasp loss into one total objective function > This is what allows them to achieve those high success rates on simulators like Habitat and Isaac Gym.

Taro: What this means for autonomy is that we're moving toward systems where local motion planning isn't just about getting a path to an object, but about making sure the entire process—from landing, to aligning your arm, and finally lifting—is coordinated > It’s about the sequence being smart.

Dev: The implication is that for legged robots in industry or complex environments, this unified policy learning model provides a framework where grasp-aware reachability directly informs navigation choices > It tackles the problem of local motion planning as a coupled navigation and manipulation problem.

Rosa: So, when you think about "LMGN: A Unified Policy Learning Model for Last-Mile Grasp-Navigation," it's really about creating a single policy that handles the whole sequence from getting there to actually getting the object > It moves beyond treating those steps as independent problems.

Conclusion: Rosa: So we're wrapping up our look at this paper by Hu and colleagues on "LMGN," which is basically their unified model for legged robots to handle both navigating to a spot and actually grabbing an object.

Dev: Yeah, they put together this whole system—the representation, the hierarchy, the joint optimization—to make sure the robot doesn't just get lost or drop the item while trying to reach it.

Rosa: The big idea is that they tackle last-mile navigation and manipulation as one single problem instead of two separate ones. It’s about making sure you know exactly where to stand and which way to go before you even try the final grab.

Dev: Right, and they do this by introducing that explicit distance and directional reachability feature. That representation takes a six-dimensional grasp pose and turns it into geometric cues for navigation, like the height you need to be at and which direction you should approach from.

Rosa: And then they use a hierarchical mixture of experts policy to model the different stages of grabbing—like adjusting your stance, lining up your arm, and finally lifting the object. It's stage-aware behavior.

Dev: The real meat is that joint optimization framework they propose. They tie together the reinforcement learning loss for grasping, the navigation loss for moving around, and a grasp loss to align everything at once.

Rosa: So what does this mean outside of the simulator? If this works reliably in Habitat and Isaac Gym, it suggests a new way to make these robots handle complex tasks where they have to move and manipulate in real-time.

Dev: It changes how we think about controlling these robots; instead of optimizing leg movement separately from arm movement, you optimize them together based on the whole task.

Rosa: We're going to look at some of the specific numbers they got on success rates in our next segment.

Hubei University · Worcester Polytechnic Institute · Institute of Automation, Chinese Academy of Sciences

cs.RO

Submitted: 2026-11-05

Updated: 2026-11-05

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 83/100

The gist: Legged robots equipped with manipulators have shown significant potential in the fields of daily life and industry in complex environments <ref:2611.07515#pg2>.

Key concepts

Explicit Distance and Directional Reachability (ED-DR)
This representation converts a standard 6D grasp pose into geometric features crucial for navigation. It explicitly encodes the required grasp height, the approach direction, and the recommended relative base placement. This makes it much more useful for planning robot movement than a typical 6D pose alone.
Hierarchical MixtureofExperts (HMoE) Policy
This policy models robot behavior across distinct manipulation stages: base adjustment, pre-grasp alignment, grasp execution, and object lift. It uses semantic routing to select the most appropriate expert route for each stage, effectively modeling the decision-making process as a stage-conditioned control problem.
Joint Optimization Framework
This method optimizes the HMoE policy by simultaneously aligning three objectives: basic reinforcement learning loss (LRL), navigation loss (Lnav), and grasp loss (Lgra). This ensures coordinated performance across all aspects of the task—locomotion, reaching, and grasping—leading to better overall success rates.
Stage-Aware Behaviors
The policy is designed to model different robot behaviors corresponding to specific phases of the last-mile task. For instance, one expert handles base adjustment while another manages pre-grasp alignment. This stage modeling ensures that the robot executes the correct sequence of actions for each part of the complex operation.

Terminology

Summary

Legged robots equipped with manipulators have shown significant potential in the fields of daily life and industry in complex environments <ref:2611. p. 007515#pg2>. To address these challenges, we propose LastMile Grasp-Navigation (LMGN), a unified policy learning model for Quadruped robot loco-manipulation <ref:2611. p. 0 p. 007515#pg5>. Extensive experiments on the Habitat and Isaac Gym simulators demonstrate that LMGN consistently outperforms state-of-the-art methods, denoting that integrating grasp-aware reachability with stage-aware policy learning significantly improves the success rates of last-mile navigation and target grasp <ref:2611. p. 0 p. 007515#pg4>.

Core Innovation and Representation

An Explicit Distance and Directional Reachability (ED-DR) representation is first introduced to explicitly represent the features of grasp-compatible base placement and approach direction <ref:2611. p. 007515#pg3>. This representation converts a conventional 6D grasp pose into geometric features for the last-mile navigation, which includes the grasp height, the approach direction, and the recommended relative base placement <ref:2611. p. 207515#pg4>. The ED-DR pose is denoted as TED-DR = (pz, γ, x′, y′), where pz preserves the target grasp height <ref:2611. p. 207515#pg4>. This explicit representation is more conducive to loco-manipulation of the lastmile navigation than a conventional 6D pose which only suits grasp <ref:2611. p. 207515#pg4>.

Policy Architecture and Stage Modeling

A semantically aligned Hierarchical MixtureofExperts (HMoE) policy models stage-aware behaviors across the stages of base adjustment, pre-grasp alignment and grasp execution, and object lift <ref:2611.07515#pg5>. The HMoE expresses policy distributions in different manipulation stages, where semantically aligned Top-1 routing selects a stage-specific expert route and generates the necessary commands <ref:2611. p. 107515#pg5>. This design models decision-making for last-mile graspnavigation as a stage-conditioned semantic control problem <ref:2611. p. 307515#pg5>.

Joint Optimization Framework

A new optimization method that jointly aligns navigation and manipulation, stage-dependent expert routing, and action-level consistency is proposed to optimize the HMoE policy <ref:2611.07515#pg5>. This joint optimization objective Ltot integrates three components: the basic RL loss LRL, the navigation loss Lnav, and the grasp loss Lgra <ref:2611. p. 107515#pg4>. The total joint optimization objective is defined as Ltot = λ1LRL + λ2Lnav + λ3Lgra <ref:2611. p. 107515#pg5>.

Performance and Ablation Analysis

Extensive experiments on the Habitat and Isaac Gym simulators demonstrate that LMGN consistently outperforms state-of-the-art methods in terms of gaze and grasp success rates <ref:2611.07515#pg4>. Ablation studies confirm the contributions of the unified policy learning model, showing that removing joint optimization seriously disrupts coordination among the base, end effector, and gripper <ref:2611. p. 507515#pg8>. The full joint optimization method achieves the highest average GazeSR and GraspSR of 84. p. 647% and 53.86%, respectively <ref:2611. p. 007515#pg8>.

Conclusion

In this paper, we propose LMGN, a unified policy framework for last-mile navigation and manipulation of legged robots <ref:2611.07515#pg9>. The ED-DR representation that transforms candidate 6D grasp poses into navigation-oriented geometric cues is first introduced to explicitly express the features of graspcompatible base placement and approach direction <ref:2611. p. 607515#pg9>. We then develop a semantically aligned Hierarchical MoE policy that models stage-specific behaviors across loco-manipulation stages including base adjustment, pre-grasp alignment and grasp execution, and object lift <ref:2611. p. 607515#pg9>. To optimize the HMoE policy, we also propose a joint optimization method that contains RL optimization for grasp-navigation, alignment optimization between expert routes and ED-DR features, and actionlevel consistency <ref:2611. p. 607515#pg9>. Experimental results on Habitat and Isaac Gym show that LMGN outperforms SOTA methods in metrics of GazeSR and GraspSR, and ablation studies confirm the contributions of the unified policy learning model <ref:2611. p. 607515#pg9>.

--- Page 1 ---

LMGN: A Unified Policy Learning Model for Last-Mile Grasp-Navigation Xuenmin Hu1, Qingxue Qian1, Wei Luo1, Benyi Ma1, Bo Tang2, Long Chen3∗ 1Hubei University, Wuhan, China 2Worcester Polytechnic Institute, Worcester, MA, USA 3Institute of Automation, Chinese Academy of Sciences, Beijing

Abstract Legged robots equipped with manipulators have shown significant potential in the fields of daily life and industry in complex environments. However, existing methods mainly focus on global navigation or manipulation independently, while overlooking the last-mile navigation problem <ref:2611. p. 007515#pg2>. To address these challenges, we propose LastMile Grasp-Navigation (LMGN), a unified policy learning model for Quadruped robot loco-manipulation <ref:2611.07515#pg5>. An Explicit Distance and Directional Reachability (ED-DR) representation is first introduced to explicitly represent the features of grasp-compatible base placement and approach direction <ref:2611.07515#pg3>. We then develop a semantically aligned Hierarchical Mixtureof Experts (HMoE) policy that models stage-aware behaviors across the stages of base adjustment, pre-grasp alignment and grasp execution, and object lift <ref:2611. p. 007515#pg5>. Finally, a new optimization method that jointly aligns navigation and manipulation, stage-dependent expert routing, and action-level consistency is proposed to optimize the HMoE policy <ref:2611. p. 007515#pg5>. Extensive experiments on the Habitat and Isaac Gym simulators demonstrate that LMGN consistently outperforms state-of-the-art methods, denoting that integrating grasp-aware reachability with stage-aware policy learning significantly improves the success rates of last-mile navigation and target grasp <ref:2611.07515#pg4>.

Introduction Mobile robots that can perceive, move and manipulate are increasingly important for industrial applications, emergency response, space operations, and life services in complex environments (Bellicoso et al. p. 0 2018). Quadruped robots equipped with manipulators are especially promising because they combine terrain mobility with fine local interaction <ref:2611. p. 007515#pg4>. Recent studies on quadruped robots equipped with manipulators have made substantial progress in navigation, obstacle traversal, and whole-body control (Wang et al. p. 0 2025). However, in practical mobile manipulation, task success is not determined only by entering the target neighborhood <ref:2611. p. 007515#pg4>. The last few steps before grasping are often decisive, which is not only a terminal correction to a path, but also a coupled local decision problem in which the robot must choose an approach direction, assess whether the current stance supports the grasp, decide whether to move the base or the arm first, and recover after a failed grasp <ref:2611. p. 007515#pg4>. This is the issue of “last-mile navigation", whose key question is how local motion should be planned to support the final grasp <ref:2611.07515#pg4>. Most previous studies related to last-mile navigation focus on local approach (Jauhri, Lueth, and Chalvatzaki 2024), base placement (Makhal and Goins 2018; Shao et al. p. 0 26). These methods improve local correction near the target, but several limitations remain for legged robots with manipulators <ref:2611. p. 007515#pg4>. Firstly, navigation and manipulation are often modeled and optimized separately <ref:2611. p. 007515#pg4>. Secondly, existing policies often lack explicit models of distance and directional reachability in the last mile <ref:2611. p. 007515#pg4>. Thirdly, the loco-manipulation process is strongly stage-dependent <ref:2611. p. 007515#pg4>.

--- Page 2 ---

Related Work Legged Loco-Manipulation Legged loco-manipulation that combines the terrain mobility of quadrupeds with the local manipulation capability of robotic arms demonstrates considerable potential in industry <ref:2611.07515#pg3>. Modular methods that decompose the legged locomanipulation process into separate components including locomotion, perception, planning, and manipulation (Bellicoso et al. p. 1 2019; Yokoyama et al. p. 1 2024; Wang et al. p. 0 2025) can leverage well-established task-specific controllers and simplify system development <ref:2611. p. 107515#pg4>. Unified methods study integrated control and long-horizon quadruped manipulation and achieved good coordination of whole-body control by optimizing the policies of leg, torso, and robotic arm (Fu, Cheng, and Pathak 2022; Yang et al. p. 1 2023; Liu et al. p. 1 2024) <ref:2611. p. 007515#pg4>. Compared to modular methods, unified methods effectively improve mobility and coordination, but control policies of different parts of the robots, such body motion and arm manipulation are independently optimized with different objectives <ref:2611. p. 107515#pg4>. The closest baseline to our method is Visual Whole-Body Control (VBC) (Liu et al. p. 1 2024), which adopts a hierarchical framework, where a high-level policy predicts the robot base velocity and end-effector pose commands from visual observations, while a low-level whole-body controller tracks these commands for object grasp <ref:2611. p. 107515#pg4>. However, unlike VBC, our method focuses on last-mile navigation for target grasp, where robot base placement, approach direction, and grasp execution are coupled through reachability-aware features and structured multi-stage policies <ref:2611. p. 107515#pg4>.

--- Page 3 ---

Problem Formulation The last-mile grasp-navigation is a local navigation for the target grasp after the robot enters the target neighborhood <ref:2611.07515#pg4>. The objective is not the global path search, but the joint choices of base placement, approach direction, and endeffector action within a limited local workspace <ref:2611. p. 107515#pg4>. Therefore, we formulate the last-mile grasp-navigation as a cooperative navigation and manipulation problem rather than a local position-correction problem <ref:2611. p. 107515#pg4>. Our method is based on reinforcement learning (RL) (Hu et al. p. 1 2026a). The last-mile navigation policy aims to select an action to optimize a comprehensive loss Ltot that contains three components: the basic RL loss LRL, the navigation loss Lnav, and the grasp loss Lgra, as shown by Eq. p. 1 1 <ref:2611. p. 007515#pg4>.

--- Page 4 ---

Method As shown in Fig. 1, the proposed LMGN takes local observations, target pose, and robot proprioceptions as inputs <ref:2611. p. 107515#pg5>. GSNet (Wang et al. p. 1 2021) is first used to generate candidate 6D grasp poses from the object point cloud, and then the proposed ED-DR representation converts them into a grasp-guided representation that includes the grasp height, the approach direction, and the recommended relative base placement <ref:2611. p. 107515#pg5>. Meanwhile, a pretrained PointNet (Qi et al. p. 1 2017) is used to extract the surrounding features of the environmental point cloud <ref:2611. p. 107515#pg5>. Because a feasible grasp also depends on the surrounding geometry and the current robot configuration, the policy backbone that consists of a three-layer multilayer perceptron (MLP) fuses these features with the target pose and robot proprioception <ref:2611. p. 107515#pg5>. In addition, since the stages of approach, pre-grasp alignment and grasp execution, and the lift or retry require different behaviors, we develop a hierarchical MoE to express policy distributions in different manipulation stages <ref:2611. p. 107515#pg5>.

--- Page 5 ---

PointNet Object Pose proprioception Hierarchical MoE Grasp Pose Top-1 Router Stage 1 approach and base adjustment Expert1 Stage 2 pre-grasp alignment and grasp Expert2 Stage 3 object lift and retry Top-1 selected router Non-selected router Base Velocity output Expert1 Expert2

--- Page 6 ---

Figure 1: Overview of the proposed LMGN. The policy fuses grasp-guided reachability features, point-cloud observations, object pose, and proprioception, then uses a geometry-aligned hierarchical MoE policy to select stage-specific expert routes for base motion, end-effector adjustment, and gripper control <ref:2611. p. 207515#pg5>.

--- Page 7 ---

Grasp-Guided Navigation Feature Representation In the last-mile grasp-navigation of robots, the policy needs to infer where the robot should stand, which direction it should approach from, and whether the current base placement allows the grasp to be executed <ref:2611.07515#pg5>. Inspired by the low-dimensional pose representation (Rudorfer 2025), we introduce an Explicit Distance and Directional Reachability (ED-DR) representation in this paper, which converts a conventional 6D grasp pose into geometric features for the last-mile navigation <ref:2611. p. 2 p. 207515#pg5>. This representation explicitly contains the appropriate stand position, approach direction and grasp height for grasping the target <ref:2611. p. 2 p. 207515#pg5>.

--- Page 8 ---

Figure 2: ED-DR representation. x′ y′ = cos γ sin γ − sin γ cos γ − px − py (Equation 3)

--- Page 9 ---

Grasp-Guided Navigation Feature Representation In the last-mile grasp-navigation of robots, the policy needs to infer where the robot should stand, which direction it should approach from, and whether the current base placement allows the grasp to be executed <ref:2611.07515#pg5>. Inspired by the low-dimensional pose representation (Rudorfer 2025), we introduce an Explicit Distance and Directional Reachability (ED-DR) representation in this paper, which converts a conventional 6D grasp pose into geometric features for the last-mile navigation <ref:2611.07515#pg5>. This representation explicitly contains the appropriate stand position, approach direction and grasp height for grasping the target <ref:2611.07515#pg5>.

--- Page 9 ---

The policy fuses grasp-guided reachability features, point-cloud observations, object pose, and proprioception, then uses a geometry-aligned hierarchical MoE policy to select stage-specific expert routes for base motion, end-effector adjustment, and gripper control <ref:2611.07515#pg5>.

--- Page 9 ---

The policy fuses grasp-guided reachability features, point-cloud observations, object pose, and proprioception then uses a geometry-aligned hierarchical MoE policy to select stage-specific expert routes for base motion, end-effector adjustment, and gripper control <ref:2611.07515#pg5>.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Unified Policy Learning Model for Last-Mile Grasp-Navigation (LMGN) can jointly determine base placement, approach direction, and grasp execution by integrating an explicit Distance and Directional Reachability (ED-DR) representation into the last-mile navigation policy.

  2. The Hierarchical Mixture of Experts (HMoE) policy models stage-aware behaviors across different manipulation stages—approach and base adjustment, pre-grasp alignment and grasp execution, and object lift—by using semantically aligned Top1 routing to select stage-specific experts.

  3. A joint optimization method is proposed to align navigation with manipulation by optimizing three components: RL optimization for last-mile grasp-navigation, alignment optimization between high-level expert routes and ED-DR features, and action-level consistency.

  4. The reward function is designed as rt = r task t + λnavr nav t + λgrar gra t, where r task t provides feedback on grasp success while navigation and grasp rewards are calculated separately to balance positional progress with grasp reachability.

  5. Action-level consistency optimization ensures command compatibility by defining the loss Lac = w g t αb (1 - cos∆bt, ∆b∗t) + w g t αg (∆T g t −∆T g∗ t) / 2 + αc LBCE c g t, c∗ t.

Sources

Related papers