World-Model-Augmented Visual Locomotion for Humanoids on Foothold-Constrained Terrain

arXiv:2609.02542 · cs.RO · Submitted 2026-09-02 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "World-Model-Augmented Visual Locomotion for Humanoids on Foothold-Constrained Terrain".

Dev: Here's the gist of the paper. Foothold-constrained terrain is characterized by sparse, discontinuous, or geometrically restricted feasible foot contacts, as encountered on stepping stones, across gaps, and on narrow stair treads <ref:2609.02542#pg2> On such terrain, a single misstep often leaves little room to recover, so policies that base foot-placement decisions primarily on the immediately visible terrain are prone to failure <ref:2609.02542#pg2>.

Rosa: The paper presents World-ModelAugmented Visual Locomotion (WM-LOCO), which jointly trains a recurrent world model and a PPO policy <ref:2609.02542#pg2>.

Dev: The Proposed Framework WM-LOCO introduces a framework that couples a recurrent world model with PPO to provide a predictive recurrent feature <ref:2609.02542#pg2>.

Rosa: The world model consumes proprioception and an egocentric depth image, maintaining a recurrent state that summarizes near-future observations and rewards <ref:2609.02542#pg2>.

Title and authors: Rosa: So, we're looking at this paper now called "World-Model-Augmented Visual Locomotion for Humanoids on Foothold-Constrained Terrain." Basically, it’s about a humanoid robot trying to walk across really tricky surfaces like stepping stones or narrow stairs where there aren't many places you can actually put your feet.

Dev: It’s interesting because the title immediately flags the problem: foothold-constrained terrain. The authors are asking if we can use some kind of learned memory, a world model, to give the robot foresight when it only sees what’s right in front of it.

Taro: I'm thinking about that lack of recovery room. If you slip on a step, you don't have much time to correct your balance before you fall off the edge. That makes me wonder if having some predictive feature could help at all, even if the terrain is really sparse and discontinuous.

Rosa: Exactly that’s the core question they’re tackling. They are suggesting that a learned summary of what happened just before might give the policy a head start on where to put its next foot, instead of just reacting instantly to what it sees right now.

Dev: It's about moving from purely reactive control, which is so brittle on these surfaces, to something that incorporates temporal context through this world model.

Taro: So we’re talking about adding a layer of prediction on top of the visual input and proprioception to manage those moments where a single bad decision can end the run.

The paper's summary: Rosa: Okay, so what the paper actually proposes is something called World-ModelAugmented Visual Locomotion, or WM-LOCO. It’s a joint training system where they train both this recurrent world model and the actual locomotion policy using PPO.

Dev: The world model itself takes in proprioception data, which is like the robot's internal sensors—joint positions, velocities—and an egocentric depth image from its camera to build this recurring state. That state summarizes everything that’s happened recently and predicts what might happen next in terms of observations and rewards.

Taro: So the world model acts like a short-term memory for the robot, summarizing the near future based on past actions and current sensor readings. That seems like a solid way to handle things that aren't just instantaneous inputs.

Rosa: Right, and this summary is then used as a feature, which they call fWMt, that conditions the policy. So instead of just seeing the immediate terrain, the policy sees both what it sees now and what it expects based on that summarized history.

Dev: The PPO policy then takes that history along with its current velocity command and the visual input to decide where to move next, leveraging this predictive feature for better decisions than a purely reactive system.

Taro: So it's like giving the decision-making part of the robot a short-term forecast based on its memory, which should help it anticipate when those sparse footholds are coming up.

The paper's improvements: Rosa: The authors aren’t just proposing this; they’re actually using specific training techniques to make it work better on these hard surfaces. They introduce some reward shaping terms that go beyond just tracking velocity.

Dev: They add penalties like a stair-boundary penalty and a riser-penetration penalty to discourage the robot from veering around the terrain instead of using the actual footholds available, which is crucial for those narrow treads or stones.

Taro: And they also have this sequential stair-tread contact reward that keeps track of an ordered sequence of where the robot lands on a staircase, ensuring it moves in a sensible direction along that structure. That’s more nuanced than just hitting any spot.

Rosa: Plus, they have a separate reward for stepping stones that activates when the robot is at the top of a tile, which helps reinforce those specific contact points on those kinds of surfaces. They even have this parallel formulation for the stepping-stone top-contact reward to make sure those valid regions are recognized correctly.

Dev: So they’ve made sure that the training objective isn't just about getting velocity right; it’s actively steering the robot to use the correct, constrained footholds by giving it specific feedback on how to interact with stairs and stones.

Conclusion: Rosa: So, wrapping things up, this World-ModelAugmented Visual Locomotion paper shows that combining a recurrent world model with PPO can handle those sparse, discontinuous terrains we talked about. They tested it across stepping stones, gaps, and stairs.

Dev: The key result is that the baseline policy fails completely on gaps and stepping stones—it hits zero percent success rate at every difficulty level there—but WM-LOCO manages to get success rates between seventy eight point two percent for the stepping stones at a hard setting and even one hundred percent on gaps at a medium setting.

Taro: That gap in performance is huge, especially when you look at those difficult environments. I'm also interested in how the gait quality actually changes; they found that WM-LOCO results in longer strides—fifteen to thirty-five percent longer—and fewer steps per meter, which suggests a more efficient movement pattern overall.

Rosa: And they showed this isn’t just about success rates; it’s also about making the robot move better on stairs too, where the improvement is more modest but still present. They achieved success rates between ninety two point zero percent and ninety five point seven percent on stairs, compared to eighty-seven point zero to ninety one point four percent for the baseline.

Dev: The deployment aspect is also strong because they ran this policy fully onboard a Jetson Orin using only proprioception and a single depth stream. This means it works in the real world without needing any complex external perception systems or precomputed maps, which is really important for practical use.

Taro: It’s good that the simulation results match what they saw on hardware—the robot actually navigates those gaps and stairs in a way similar to how it was simulated, which gives confidence in the model.

Rosa: So this World-ModelAugmented Visual Locomotion paper gives us a framework where we can train robots to walk on challenging, real-world surfaces using only onboard sensors and an onboard computer. That’s the big picture here.

Yuxi Liu, Lijun Han, Ziming Wang, Ao Zhang, Cong Yang, Wei Sui

D-Robotics Institute of Robotics and Automation Research Department Beijing University of Posts and Telecommunications Harbin Institute of Technology Soochow University

cs.RO

Submitted: 2026-09-02

Updated: 2026-11-03

Comments: 8 pages, 6 figures, 4 tables. Yuxi Liu and Lijun Han contributed equally

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 83/100

The gist: Foothold-constrained terrain is characterized by sparse, discontinuous, or geometrically restricted feasible foot contacts, as encountered on stepping stones, across gaps…

Key concepts

World Model
A recurrent system that learns how the environment works by processing past actions and current observations. It maintains an internal state summarizing near-future predictions, allowing the robot to anticipate what will happen next based on its history.
Recurrent Feature (fWMt)
A predictive feature derived from the world model's internal state. This feature conditions the policy, giving it anticipatory information about future states and terrain features that are not immediately visible, which is vital for navigating sparse footholds.
PPO Policy
The reinforcement learning policy that decides on actions (like foot placement). In WM-LOCO, this policy takes the robot's history, current movement commands, and the predictive feature from the world model to choose actions more intelligently.
Foothold-Constrained Terrain
Environments where feasible foot contacts are sparse or geometrically restricted, such as stepping stones or narrow stair treads. These terrains challenge robots because a single misstep can lead to failure, requiring more sophisticated planning than simple reactive control.

Terminology

Summary

Foothold-constrained terrain is characterized by sparse, discontinuous, or geometrically restricted feasible foot contacts, as encountered on stepping stones, across gaps, and on narrow stair treads <ref:2609. p. 002542#pg2> On such terrain, a single misstep often leaves little room to recover, so policies that base foot-placement decisions primarily on the immediately visible terrain are prone to failure <ref:2609. p. 002542#pg2>. The paper presents World-ModelAugmented Visual Locomotion (WM-LOCO), which jointly trains a recurrent world model and a PPO policy <ref:2609. p. 002542#pg2>.

The Proposed Framework

WM-LOCO introduces a framework that couples a recurrent world model with PPO to provide a predictive recurrent feature <ref:2609.02542#pg2>. The world model consumes proprioception and an egocentric depth image, maintaining a recurrent state that summarizes near-future observations and rewards <ref:2609. p. 102542#pg2>. A feature fWMt derived from this recurrent state conditions the policy <ref:2609. p. 102542#pg2>. The PPO policy then takes a proprioceptive history, a velocity command, and an egocentric depth image, together with fWMt <ref:2609. p. 102542#pg2>.

World Model Formulation

The world model processes pt and dt using a recurrent state-space model (RSSM) [12, 13] with a deterministic memory ht and a stochastic latent zt (dimension 128) <ref:2609.02542#pg2>. The model learns (i) a transition ht = fϕ(ht−1, zt−1, at−1) that updates the memory with the last action, (ii) a posterior zt ∼ qϕ(zt ht, pt, dt) that infers zt from the memory and the current observation, (iii) a prior zˆt ∼ pϕ(zt ht) that predicts zt from the memory without the current observation, and (iv) decoders that reconstruct proprioception, depth, and reward <ref:2609. p. 202542#pg2>. The training objective is LWM = MSE(ˆpt, pt)z proprio recon + MSE(ˆdt, dt)z depth recon + MSE(ˆrt, rt)z reward pred +β KL(qϕ ∥ pϕ) z forward prediction <ref:2609. p. 202542#pg2>.

Training Reward Shaping

A standard locomotion objective—velocity tracking plus standard gait shaping—is too sparse for foothold-constrained terrain <ref:2609.02542#pg2>. The authors augment the standard reward rstdt with terrain-specific shaping terms, applied identically to WM-LOCO and PPO within each terrain-training setting <ref:2609. p. 302542#pg2>. These include:

  1. Stair-boundary penalty (rbdry t) which discourages detouring around the terrain to satisfy velocity tracking without engaging the foothold constraint <ref:2609. p. 302542#pg2>.

  2. Riser-penetration penalty (rriser t) which counts foot sample points that fall inside any stair riser, preventing oblique foot contacts near the leading edge of the tread from being misclassified as riser penetrations <ref:2609. p. 302542#pg2>.

  3. Sequential stair-tread contact reward (rtread t) which maintains an ordered sequence of horizontal stair-tread contact regions R1, R2,. p. 3.. in the robot’s current heading <ref:2609. p. 302542#pg2>.

  4. Stepping-stone top-contact reward which uses a separate parallel formulation where all valid stone-top regions for the current tile are active at reset <ref:2609. p. 302542#pg2>.

Experimental Results

Table 1 reports success rates at three difficulty levels: Easy, Medium, and Hard <ref:2609.02542#pg2>. On gaps and stepping stones the performance difference is substantial: the PPO baseline reaches 0. p. 40% success at every level, while WM-LOCO reaches between 78. p. 42% (stepping stones, Hard) and 100% (gap, Medium) <ref:2609. p. 402542#pg2>. On stairs, PPO achieves success rates between 87. p. 40% and 91.4%, whereas WM-LOCO achieves rates between 92. p. 40% and 95.7% <ref:2609. p. 002542#pg2>. Overall, the performance gap is modest on stairs but widens on gaps and stepping stones, where PPO reaches 0% at every difficulty tier while WM-LOCO remains above 78% <ref:2609. p. 402542#pg2>.

Gait Quality Improvements

On stairs both methods reach similar success rates, but they differ in gait quality <ref:2609.02542#pg2>. WM-LOCO exhibits longer strides (stride length +15% to +35%), fewer steps per meter (−9% to −21%), and lower pelvis acceleration (−24% to −33%) <ref:2609. p. 402542#pg2>. It also reduces normalized mechanical energy consumption by 6%–20% <ref:2609. p. 402542#pg2>.

Sim-to-Real Validation

The policy runs fully onboard a Jetson Orin from proprioception and a single headmounted depth stream <ref:2609.02542#pg2>. On stepping stones with 0. p. 025 m-wide top surfaces and 0. p. 645 m longitudinal gaps, the policy successfully traverses the discrete pads without observed foot-placement failures <ref:2609. p. 602542#pg2>. On stairs with 0. p. 015 m risers and 0. p. 625 m treads, the robot negotiates each riser and places each foot flat on the tread without additional tuning <ref:2609. p. 602542#pg2>. The robot traverses the 0. p. 18 m gap in a single step, exhibiting a swing-and-landing pattern similar to that observed in simulation <ref:2609. p. 602542#pg2>.

Conclusion

We present WM-LOCO, a visual humanoid locomotion framework for foothold-constrained terrain <ref:2609.02542#pg2>. A recurrent world model, jointly trained with PPO, supplies a predictive recurrent feature to the actor–critic using onboard proprioception and depth <ref:2609. p. 602542#pg2>. On the evaluated stepping-stone and gap classes, WM-LOCO achieves high success rates, whereas the PPO baseline records no successful trials <ref:2609. p. 602542#pg2>. The same policy runs onboard a Jetson Orin <ref:2609. p. 602542#pg2>. In real-time hardware experiments, the robot exhibits contact patterns similar to those observed in simulation across all three terrain classes <ref:2609. p. 602542#pg2>.

References

[1] N. Rudin, D. Hoeller, P. p. 6 Reist, and M. p. 6 Hutter Learning to walk in minutes using massively parallel deep reinforcement learning In Proceedings of the 5th Conference on Robot Learning volume 164 of Proceedings of Machine Learning Research pages 91–100 PMLR, 2022 <ref:2609. p. 602542#pg9>

[3] H. Wang, Z. Wang, J. Ren, Q. Ben, T. Huang, W. Zhang, and J Pang BeamDojo Learning agile humanoid locomotion on sparse footholds In Proceedings of Robotics: Science and Systems (RSS), 2025 doi:10. p. 615607/RSS.2025.XXI.068

[4] Y Zhang, Y Seo, J Chen, Y Yuan, K Sreenath P Abbeel C Sferrazza K Liu R Duan and G Shi RPL Learning robust humanoid perceptive locomotion on challenging terrains arXiv preprint arXiv:2602.03002 2026

[8] H Lai, J Cao, J Xu, H Wu, Y Lin, T Kong, Y Yu, and W Zhang World model-based perception for visual legged locomotion In IEEE International Conference on Robotics and Automation ICRA pages 11531–11537 2025 doi:10.1109/ICRA55743. p. 7 p. 72025.11128762

[9] W Sun, L Chen, Y Su, B Cao, Y Liu, and Z Xie Learning humanoid locomotion with world model reconstruction arXiv preprint arXiv:2502.16230 2025

[11] Y Fan, T Gui, K Ji, S Ding, C Zhang, Y Xu, K Yang, J Gu, J Yu J Wang and Y Shi DreamPolicy A unified world-model policy for scalable humanoid locomotion arXiv preprint arXiv:2505.18780 2025

[13] D Hafner, T Lillicrap, J Ba, and M Norouzi Dream to control Learning behaviors by latent imagination In International Conference on Learning Representations ICLR 2020

[14] S Zhu, Z Zhuang, M Zhao, K.-Y Lee, and H Zhao Hiking in the wild A scalable perceptive parkour framework for humanoids arXiv preprint arXiv:2601. p. 7 p. 707718 2026

[15] R Yu, Q Wang, H Li, Z Jun, Z Wang, J Wu, and Q Zhu START Traversing sparse footholds with terrain reconstruction arXiv preprint arXiv:2512.13153 2025

[6] M Dai, W D Compton, J Li, L Yang, and A D Ames Walk the PLANC Physics-guided RL for agile humanoid locomotion on constrained footholds arXiv preprint arXiv:2601.06286 2026

[7] Z Xiang, U Pant, and A Hereid Perceptive variable-timing footstep planning for humanoid locomotion on disconnected footholds arXiv preprint arXiv:2603.07400 2026

[12] D Hafner, J Pasukonis, J Ba, and T Lillicrap Mastering diverse control tasks through world models Nature 640:647–653 2025 doi:10.1038/s41586-025-08744-2

[13] D Hafner, J Pasukonis, J Ba, and T Lillicrap Mastering diverse control tasks through world models Nature 640:647–653 2025 doi:10.1038/s41586-025-08744-2

[1] N Rudin, D Hoeller, P Reist, and M Hutter Learning to walk in minutes using massively parallel deep reinforcement learning In Proceedings of the 5th Conference on Robot Learning volume 164 of Proceedings of Machine Learning Research pages 91–100 PMLR 2022

[7] Z Xiang, U Pant, and A Hereid Perceptive variable-timing footstep planning for humanoid locomotion on disconnected footholds arXiv preprint arXiv:2603.07400 2026

[8] H Lai, J Cao, J Xu, H Wu, Y Lin, T Kong, Y Yu, and W Zhang World model-based perception for visual legged locomotion In IEEE International Conference on Robotics and Automation ICRA pages 11531–11537 2025 doi:10.1109/ICRA55743.2025.11128762

[9] W Sun, L Chen, Y Su, B Cao, Y Liu, and Z Xie Learning humanoid locomotion with world model reconstruction arXiv preprint arXiv:2502.16230 2025

[11] Y Fan, T Gui, K Ji, S Ding, C Zhang, Y Xu, K Yang, J Gu, J Yu J Wang and Y Shi DreamPolicy A unified world-model policy for scalable humanoid locomotion arXiv preprint arXiv:2505.18780 2025

[13] D Hafner, T Lillicrap, J Ba, and M Norouzi Dream to control Learning behaviors by latent imagination In International Conference on Learning Representations ICLR 2020

[14] S Zhu, Z Zhuang, M Zhao, K.-Y Lee, and H Zhao Hiking in the wild A scalable perceptive parkour framework for humanoids arXiv preprint arXiv:2601.07718 2026

[15] R Yu, Q Wang, H Li, Z Jun, Z Wang, J Wu, and Q Zhu START Traversing sparse footholds with terrain reconstruction arXiv preprint arXiv:2512.13153 2025

[6] M Dai, W D Compton, J Li, L Yang, and A D Ames Walk the PLANC Physics-guided RL for agile humanoid locomotion on constrained footholds arXiv preprint arXiv:2601.06286 2026

[7] Z Xiang, U Pant, and A Hereid Perceptive variable-timing footstep planning for humanoid locomotion on disconnected footholds arXiv preprint arXiv:2603.07400 2026

[12] D Hafner, J Pasukonis, J Ba, and T Lillicrap Mastering diverse control tasks through world models Nature 640:647–653 2025 doi:10.1038/s41586-025-08744-2

[13] D Hafner, J Pasukonis, J Ba, and T Lillicrap Mastering diverse control tasks through world models Nature 640:647–653 2025 doi:10.1038/s41586-025-08744-2

[1] N Rudin, D Hoeller, P Reist, and M Hutter Learning to walk in minutes using massively parallel deep reinforcement learning In Proceedings of the 5th Conference on Robot Learning volume 164 of Proceedings of Machine Learning Research pages 91–100 PMLR 2022

[7] Z Xiang, U Pant, and A Hereid Perceptive variable-timing footstep planning for humanoid locomotion on disconnected footholds arXiv preprint arXiv:2603.07400 26

Appendix

Table 2 lists the physical dimensions of the three simulation difficulty tiers and of the real course. Table 3 lists the complete reward configurations used for the two terrain-training settings; their weights appear in Section B. p. 8

C Training and Architecture

Both methods are trained with PPO using the same hyperparameters and training budget. WM-LOCO additionally includes the RSSM pathway, its auxiliary objective, and the recurrent feature supplied to the policy. p. 8 The terrain curriculum advances independently for each environment according to exponential velocity-tracking scores. p. 8

D Additional World-Model Reconstruction Results

Figure 3 b shows that after one warm-up tick (MSE ≈ 2.5 × 10−3 on [0, 1]-normalized depth), the per-tick depth error settles into a 2–7 × 10−4 band, with a transient increase to ∼ 1. p. 82 × 10−3 near steps 45–55, when a high-relief structure crosses the field of view, and subsequently decreases to ∼ 1 × 10−4 for the rest of the 100-tick episode. p. 8 The snapshots in Figure 3 a show that the largest residuals are concentrated around raised obstacles (step 33), the top edge of the depth window (step 1), and a near-field foot target passing under the camera (step 99); smooth-floor regions reconstruct with minimal error. p. 8 Proprioceptive reconstruction exhibits lower error: after the same one-tick warm-up the total per-tick MSE drops below 5 × 10−4 and remains near this level for the rest of the rollout, with base angular velocity being the channel with the largest reconstruction error (∼ 1 × 10−3, consistent with a high-bandwidth IMU-derived signal) and projected gravity, joint position, joint velocity, and velocity command each below 5 × 10−4. p. 8

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. World-ModelAugmented Visual Locomotion (WM-LOCO) improves performance on sparse, discontinuous, or geometrically restricted footholds by jointly training a recurrent world model and a PPO policy. This system can traverse all three terrain classes—stairs, gaps, and stepping stones—with an average success rate of 93.3% on physical hardware when conditioned only on proprioception and a single depth stream.

  2. The world model provides a predictive recurrent feature fWMt that guides the policy without explicit foothold labels, addressing the limitation where policies that base foot-placement decisions primarily on the immediately visible terrain are prone to failure. This allows for anticipatory information crucial for bipeds when feasible footholds become sparse.

  3. The system can achieve improved gait efficiency and reduced physical strain on stairs by utilizing a predictive recurrent feature, as WM-LOCO improves stride efficiency and reducing pelvis acceleration compared to the baseline.

  4. The policy can be deployed fully onboard a Jetson Orin using only proprioception and a single depth stream, eliminating the need for offboard perception or precomputed terrain maps for real-time traversal of stepping stones, stairs, and gaps.

Sources

Related papers