Accelerated Reinforcement Learning for Soft Robotics: Hours Instead of Years

arXiv:2611.11718 · cs.RO, cs.AI, cs.LG, math.DS · Submitted 2026-11-09 · Read on arXiv

cs.RO, cs.AI, cs.LG, math.DS

Submitted: 2026-11-09

Updated: 2026-11-09

Comments: 25 pages, 6 figures, 3 extended data figures

Code: https://github.com/haller-group/SSMLearn

License: http://creativecommons.org/licenses/by/4.0/

The gist: Reinforcement learning (RL) requires large amounts of interaction data or a reliable simulator, both hard to obtain for nonlinear soft robotic hardware.

Terminology

Abstract

Reinforcement learning (RL) requires large amounts of interaction data or a reliable simulator, both hard to obtain for nonlinear soft robotic hardware. Recent work shows that a time-scale separation inherent to soft robot dynamics justifies their reduction to low-dimensional invariant manifolds, known as adiabatic spectral submanifolds (aSSMs). We demonstrate that an aSSM-reduced model, a simple polynomial ordinary differential equation learned purely from pre-collected data, can serve as an RL training environment. On a pneumatically actuated soft arm, we train a trajectory-tracking policy in 3 hours on a six-dimensional aSSM-reduced model and deploy it without fine-tuning, whereas reproducing this policy training extrapolates to 3 years of hardware interaction. Our policy tracks dynamic targets with 40% lower error than otherwise identical policies trained on a 4.8-million-parameter recurrent neural network (RNN) or a 1.7-million-parameter transformer, which take 13 and 100 hours to identify, against 5 minutes for our mathematically justified 6,924-parameter model.

Sources

Related papers