Beyond Topology: A Morphological Symmetry Graph Representation for Locomotion Policy Learning

summary

The gist

Reinforcement learning has enabled impressive locomotion skills on articulated robots, but common policy representations remain only weakly aligned with robot physics Starting from the robot’s…

In short

The paper introduces a morphological symmetry graph representation for learning locomotion policies on robots. It augments robot observations and actions with transformations like permutations and sign changes based on morphological symmetry. This creates a policy that respects the robot's physical structure, leading to better generalization, robustness, and efficiency compared to existing methods.

Key concepts

Morphological Symmetry Graph Representation
This method models a robot as a graph where nodes are joints and edges are connections. It enhances this graph by including actions and observations that account for the robot's inherent morphological symmetry, such as reflections or sign changes. This ensures the learned policy respects the physical structure of the robot.
Symmetry-Equivariant Graph Actor
The actor in this system is designed to be symmetry-equivariant. This means if you input an observation that has been mirrored or transformed by a morphological symmetry operation, the resulting action will also be mirrored or transformed in a corresponding way. It enforces physical consistency during policy learning.
Symmetry-Invariant Graph Critic
The critic is designed to be symmetry-invariant. This means that if two different states are morphologically symmetric—meaning one can be transformed into the other by a symmetry operation—the critic assigns them the same value. This helps the system learn a more consistent understanding of robot states.
Policy Equivariance vs. Value Invariance
The actor enforces policy equivariance, ensuring mirrored inputs yield mirrored outputs. The critic enforces value invariance, ensuring that morphologically symmetric states share the same predicted value. These properties are built into the representation to guide the learning process physically.

Terminology used across episodes

This episode discusses

The paper

Beyond Topology: A Morphological Symmetry Graph Representation for Locomotion Policy Learning · Read on arXiv

Sizhe Wei, Xulin Chen, Fengze Xie, Garrett Ethan Katz, Zhenyu Gan, Lu Gan

Georgia Institute of Technology · California Institute of Technology

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: I'm Rosa.

Dev: And I'm Dev, here with Taro.

Rosa: We're looking at the paper: "Beyond Topology: A Morphological Symmetry Graph Representation for Locomotion Policy Learning".

Dev: Here's the gist of the paper. Reinforcement learning has enabled impressive locomotion skills on articulated robots, but common policy representations remain only weakly aligned with robot physics Starting from the robot’s topological graph, our representation augments each observation and action space with the permutation and sign transformations induced by morphological symmetry This yields a symmetry-equivariant graph actor and a symmetry-invariant graph critic, enforcing the desired policy and value constraints by construction rather than through reward shaping or data augmentation Experiments show improved symmetry generalization, robustness, sample efficiency, and model efficiency over topology- and symmetry-aware baselines The Core Representation We introduce a morphological symmetry graph representation for locomotion policy learning We represent an articulated robot as a graph and complete this graph with morphological symmetry actions on observations, privileged observations, and actions, including the permutations and sign changes induced by sagittal reflection Built on this representation, MS-PPO uses a morphological-symmetry-equivariant graph actor and a morphologicalsymmetry-invariant graph critic The goal is not to define a new RL optimizer, but to define a physically aligned function space in which policy and value functions respect both the robot’s kinematic structure and its morphological symmetry How the Training Signal is Built The representation involves augmenting the topological graph with morphological symmetry actions on observations, privileged observations, and actions The actor identity is policy equivariance, meaning mirrored observations produce mirrored actions The critic identity is value invariance, meaning morphologically symmetric states have the same value This is achieved by defining the actor as policy equivariance, where mirrored observations produce mirrored actions <ref:2512.00727#pg2>, and the critic as value invariance, meaning morphologically symmetric states have the same value <ref:2512.00727#pg5> The encoder h(·) maps physical robot signals into graph features that transform only by graph reindexing under G, as in Eq.

Rosa: (3) Where the Gains Come From Experiments show improved symmetry generalization, robustness, sample efficiency, and model efficiency over topology- and symmetry-aware baselines MS-PPO consistently outperforms strong morphology- and symmetry-aware baselines, achieving the best joint-failure tolerance, the lowest sim-to-real velocity MSE on G1 with only 34.6% of standard PPO’s parameters <ref:2512.00727#pg7> The results show that jointly encoding kinematic topology and morphological symmetry is an effective representation design for locomotion policy learning on articulated robots MS-PPO uses only 34.6% of the parameters of PPO and PPO-Aug and 69.3% of the parameters of PPO-EMLP, suggesting that jointly encoding kinematic connectivity and morphological symmetry improves model efficiency without sacrificing task-level performance <ref:2512.00727#pg7> Evaluation Across Embodiments We evaluate MS-PPO on the Unitree Go2 quadruped and G1 humanoid robots MS-PPO achieves the best velocity tracking among all deployable methods, while PPO and MI-PPO fail to produce stable hardware locomotion <ref:2512.00727#pg8> These results support our central claim that effective morphology-aware policy learning requires encoding symmetry structure consistently within the robot’s topological graph representation <ref:2512.00727#pg8> MS-PPO improves symmetry generalization, robustness to asymmetric joint failures, sample efficiency, model efficiency, and zeroshot sim-to-real deployment relative to topology-aware and symmetry-aware baselines Limitations Our current validation is limited to bilateral C2 symmetry on one quadruped and one humanoid <ref:2512.00727#pg9> We have not yet tested richer or approximate symmetry structures, such as rotational Cn robots, reflective K4 quadrupeds, radially arranged multi-arm systems, or symmetric manipulators <ref:2512.00727#pg10> Another limitation is that the current experiments instantiate the representation separately on each robot, rather than demonstrating policy transfer across embodiments with different node features, action spaces, and symmetry orbits <ref:2512.00727#pg10> Future work will address these limitations by automating symmetry graph construction and symmetry action deriving <ref:2512.

Paper summary: Dev: So, wrapping up this look at Beyond Topology: A Morphological Symmetry Graph Representation for Locomotion Policy Learning. The core message here is that the way we represent a robot's body matters more than just connecting the joints; we need to explicitly encode how physical properties change when you reflect or mirror parts of that body.

Rosa: Yeah, it moves beyond just using the robot's topology as a starting point and instead augments it with these symmetry actions to create a representation that is both policy-equivariant and value-invariant <ref:two thousand five hundred twelve point zero zero seven two seven#pg3. This means the AI learns policies that respect the robot's physical structure, not just its connectivity, which helps it be more robust in the real world.

Taro: For someone looking at this from a deployment standpoint, what this really means is that we are building a system where the underlying mathematical structure of the robot’s shape is baked into the learning process itself rather than being an afterthought. It’s about making sure that when you deploy it on something new, or something slightly different, it has a much better chance of working because it already understands the physics of symmetry.

Dev: Exactly. And the results are compelling, showing improved efficiency and performance compared to existing topology-aware and symmetry-aware methods <ref:two thousand five hundred twelve point zero zero seven two seven#pg7. It shows that jointly encoding kinematic topology and morphological symmetry is an effective design choice for locomotion policy learning on articulated robots.

Rosa: So, while they’ve made a strong case for this representation, the authors are being clear about where they still need to go, specifically needing to test it across more complex or approximate symmetry structures <ref:two thousand five hundred twelve point zero zero seven two seven#pg10. It's a solid step forward in alignment between policy and robot physics.

Conclusion: Rosa: So, this paper introduces something called Beyond Topology: A Morphological Symmetry Graph Representation for Locomotion Policy Learning. It’s about taking the basic graph structure of a robot and adding information about its shape's symmetry—how it looks when you flip or mirror parts of it.

Dev: Right, so instead of just focusing on how the joints connect, they are augmenting the observation and action spaces with those permutation and sign transformations caused by morphological symmetry. That means the AI learns policies that respect the robot’s actual physical shape, not just its wiring diagram.

Taro: From my side, what this implies is that we’re building a system where the physical rules of how a robot moves are baked directly into how it learns to walk or run in real-time. If you can teach it symmetry equivariance and invariance at the representation level, it should generalize better when things get messy outside the lab.

Rosa: Exactly. They show that using this setup, they get a graph actor that is equivariant—meaning if you mirror the robot's observation, the action mirrors too—and a graph critic that is invariant—meaning symmetric states have the same value. It enforces these physical constraints by construction instead of relying on tweaking reward functions or adding extra data.

Dev: And the numbers back up what they’re saying, though they keep things focused. They found better symmetry generalization, more robustness when joints fail in asymmetric ways, and improved sample efficiency compared to existing topology-aware and symmetry-aware methods. It even uses much less of the policy parameters than some other complex setups.

Taro: That efficiency is key for deployment because it means we don't need massive amounts of data just to get a stable gait. But I have to point out their limitations, they only tested this on very specific, bilateral C2 symmetry in one quadruped and one humanoid robot right now.

Rosa: True. And another thing they noted is that they built this representation separately for each robot instead of showing how the policy transfers when you switch between robots with different features or action spaces. That's a big gap for deployment across different platforms, right?

More episodes

← Home