Learning Soft Robotic Dynamics with Active Exploration

arXiv:2510.27428 · cs.RO, cs.AI · Submitted 2025-10-31 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: I'm Rosa.

Dev: And I'm Dev, here with Taro.

Rosa: We're looking at the paper: "Learning Soft Robotic Dynamics with Active Exploration".

Dev: Here's the gist of the paper. Soft robots offer unmatched adaptability and safety in unstructured environments, yet their compliant, high-dimensional, and nonlinear dynamics make modeling for control notoriously difficult SOFTAE introduces an uncertainty-aware active exploration framework that autonomously learns task-agnostic and generalizable dynamics models of soft robotic systems Problem Addressed Soft robots exhibit complex, high-dimensional dynamics due to continuous deformation, distributed compliance, and nonlinear material properties These systems lack well-defined coordinates or rigid links, rendering traditional kinematic and dynamic formulations inapplicable or imprecise Existing data-driven approaches are limited by the quality and diversity of training data.

Rosa: Task-specific data collection often results in narrow, overfitted models that generalize poorly beyond the training distribution In such settings, retraining models for each new objective is impractical due to the cost and time associated with the collection of real-world data.

Dev: Objective The objective of this work is to learn task-agnostic dynamics models for soft robots that generalize across morphologies, actuation regimes, and environmental conditions The authors propose an active exploration framework that uses uncertainty estimates to autonomously guide data collection toward underexplored regions of the state–action space.

Rosa: The goal is to build a model that captures the full behavioral range of the robot and can be used for zero-shot planning and control across a variety of downstream tasks without retraining or task-specific adaptation.

Paper summary: Rosa: So, we're looking at this paper called "Learning Soft Robotic Dynamics with Active Exploration." Essentially, they're tackling the problem that soft robots are super adaptable and safe in messy situations, but modeling their movement is incredibly hard because of all that continuous deformation and distributed compliance <ref:2510.27428#pg2>.

Dev: Right. The core issue is that traditional math doesn't work well for these systems because they don't have neat coordinates or rigid links, so you can't just use the same old kinematic formulas <ref:2510.27428#pg3>. Existing data-driven methods are also really limited because they struggle to get enough diverse training data, often ending up with models that only work for the specific task they were trained on <ref:2510.27428#pg3>.

Rosa: So what's the idea here? They introduce SOFTAE, which is this uncertainty-aware active exploration framework <ref:2510.27428#pg1>. The thesis is that instead of just collecting random data, they use model uncertainty as a signal to tell the robot exactly where it needs to go next in terms of state and action space <ref:2510.27428#pg1>.

Dev: They use probabilistic ensemble models to estimate this epistemic uncertainty, which basically means they're trying to figure out how unsure the model is about its own predictions at a certain point <ref:2510.27428#pg1>. The goal is to autonomously guide data collection toward those underrepresented areas of behavior that the current model doesn't understand well <ref:2510.27428#pg1>.

Taro: I'm interested in how it handles the complexity, because soft robots are nonlinear and coupled in ways that are really non-intuitive <ref:2510.27428#pg3>. So if the uncertainty guides exploration, does it actually find those complex behaviors that a random search would miss?

Rosa: Exactly what I want to know. They show this approach learns dynamics models that are task-agnostic and generalizable across different robot shapes and actuation styles <ref:2510.27428#pg1>. This means you train it once, and it should be able to plan for totally new tasks without needing retraining or specific data for each one <ref:2510.27428#pg1>.

Dev: That generality is key because we can't afford to collect real-world data for every single new objective, which is a huge bottleneck right now <ref:2510.27428#pg3>. They are trying to build a model that captures the full range of motion and behavior possible for the robot without being tied down by one specific goal <ref:2510.27428#pg1>.

Conclusion: Rosa: So, wrapping up this look at "Learning Soft Robotic Dynamics with Active Exploration," it seems the main point is using uncertainty to drive how we learn the dynamics of soft robots <ref:2510.27428#pg1>. They propose this active exploration framework to build models that generalize across different robot shapes and actuation regimes <ref:2510.27428#pg1>.

Dev: And what that means for control is that you get these task-agnostic models, which should allow for zero-shot planning onto previously unseen control objectives <ref:2510.27428#pg1>. It shifts the focus from gathering task-specific data to intelligently exploring the state and action space to build a robust model <ref:2510.27428#pg3>.

Taro: For real-world deployment, that seems important because it suggests we could deploy these systems in unpredictable environments where we don't know exactly what challenges they'll face <ref:2510.27428#pg1>. It’s about the system being able to handle when things go wrong because the model is already trained on a wide variety of behaviors <ref:2510.27428#pg3>.

Rosa: Right. So, we're moving toward systems that can learn how to behave broadly across many scenarios without having to be hand-coded for every single one <ref:2510.27428#pg1>. It’s about making the learning process itself more efficient and less dependent on perfect initial data collection <ref:2510.27428#pg3>.

Dev: That's the gist of it. The paper shows that by targeting regions where the model is least certain, we get better coverage and ultimately more accurate dynamics models across the entire reachable space <ref:2510.27428#pg3>. This method seems to offer a way forward for making soft robot control more practical in complex settings <ref:2510.27428#pg1>.

Hehui Zheng, Bhavya Sukhija, Chenhao Li, Klemens Iten, Andreas Krause, Robert K Katzschmann

Soft Robotics Lab, ETH Zurich · Learning & Adaptive Systems Group, D-INFK, ETH Zurich · ETH AI Center, ETH Zurich

cs.RO, cs.AI

Submitted: 2025-10-31

Updated: 2026-11-04

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 98/100

The gist: Soft robots offer unmatched adaptability and safety in unstructured environments, yet their compliant, high-dimensional…

Key concepts

Soft Robots Dynamics
Soft robots are complex systems where the material itself deforms continuously. Unlike rigid robots with fixed joints, their dynamics are high-dimensional, nonlinear, and lack simple coordinates. This complexity makes traditional modeling very difficult.
Active Exploration Framework (SOFTAE)
This is a method that autonomously decides where to collect data next. It uses uncertainty estimates from the current model to prioritize exploring parts of the robot's movement and control space that are currently least understood, ensuring broad coverage.
Model Uncertainty
This refers to how unsure the current learned dynamics model is about its predictions in a specific state or action scenario. High uncertainty signals an area where more data is needed to build a more accurate and reliable understanding of the robot's behavior.

Terminology

Summary

Soft robots offer unmatched adaptability and safety in unstructured environments, yet their compliant, high-dimensional, and nonlinear dynamics make modeling for control notoriously difficult SOFTAE introduces an uncertainty-aware active exploration framework that autonomously learns task-agnostic and generalizable dynamics models of soft robotic systems

Problem Addressed

Soft robots exhibit complex, high-dimensional dynamics due to continuous deformation, distributed compliance, and nonlinear material properties These systems lack well-defined coordinates or rigid links, rendering traditional kinematic and dynamic formulations inapplicable or imprecise Existing data-driven approaches are limited by the quality and diversity of training data. Task-specific data collection often results in narrow, overfitted models that generalize poorly beyond the training distribution In such settings, retraining models for each new objective is impractical due to the cost and time associated with the collection of real-world data. p. 1

Objective

The objective of this work is to learn task-agnostic dynamics models for soft robots that generalize across morphologies, actuation regimes, and environmental conditions The authors propose an active exploration framework that uses uncertainty estimates to autonomously guide data collection toward underexplored regions of the state–action space. The goal is to build a model that captures the full behavioral range of the robot and can be used for zero-shot planning and control across a variety of downstream tasks without retraining or task-specific adaptation. p. 1

Background and Related Work

Physics-based modeling approaches face trade-offs between accuracy, generality, and computational tractability Data-driven approaches are limited by the quality and diversity of training data. Active exploration in dynamics learning has been underexplored in the context of soft robotics where state and action spaces are high-dimensional, often partially observable, and coupled in complex, non-intuitive ways. p. 3

Contributions

The work makes three primary contributions toward data-efficient learning of generalizable dynamics models for soft robotic systems. First, they propose SOFTAE, an active exploration framework that combines optimism tailored to the unique challenges of soft robotics by leveraging model uncertainty as an intrinsic motivation signal to autonomously guide exploration toward regions of the state–action space where the current dynamics model is least certain Second, they demonstrate that the resulting dynamics models are task-agnostic and highly generalizable, enabling effective zero-shot planning for previously unseen control objectives. p. 3 Finally, they provide extensive empirical validation of their approach both in simulation and in real soft robotic hardware. p. 4

Results

The evaluation was conducted across three simulated systems—a continuum arm, an articulated fish in fluid, and a musculoskeletal leg with hybrid actuation—and on a pneumatically actuated continuum soft arm in the real world. SOFTAE substantially outperforms RANDOM exploration, especially in environments where effective behavior depends on coordinated actuation under nonlinear or delayed dynamics. p. 6 Across all tasks, SOFTAE matches or exceeds the performance of H-UCRL on those for which it was trained, and significantly outperforms H-UCRL on held-out tasks where H-UCRL fails to generalize. p. 6

Data Diversity Through Active Exploration

The strong downstream performance of SOFTAE suggests its advantage comes from collecting training data that is broader in coverage and more informative than baselines. This is demonstrated by showing that SOFTAE achieves far more uniform coverage across the reachable workspace compared to RANDOM and H-UCRL. p. 8 This diversity provides richer supervision for dynamics learning, which they next assess through model accuracy. p. 8

Model Accuracy with Collected Data

SOFTAE’s broader and more balanced exploration yields significantly lower next-state prediction errors across the reachable workspace compared to baselines. This indicates that by targeting regions of high uncertainty, active exploration improves data coverage and directly translates into more accurate dynamics models. p. 9

Scalability to High-Dimensional Dynamics

The method's scalability is tested by revisiting the continuum arm environment from tasks (i) and (ii) and increasing the resolution of the state representation to 136 dimensions. As shown in Figure 3C, SOFTAE maintains high performance on both reaching tasks, achieving fast learning and high final reward despite the increased dimensionality. p. 9

Real-World Active Exploration with a Pneumatically Actuated Arm

SOFTAE was validated on the pneumatically actuated SoPrA arm in the real world, demonstrating reliable performance under sensing noise, actuation delays, and complex material behaviors. In task (x), which involves reaching toward the edge of the workspace, SOFTAE outperforms both RANDOM and H-UCRL. p. 8 This consistency is critical for real-world deployment of soft robotic systems where new tasks may arise without retraining and safety margins depend on reliable worst-case performance. p. 1

Discussion

This work addresses the fundamental challenge of learning generalizable dynamics models for soft robotic systems through an active exploration framework that leverages epistemic uncertainty to guide data collection toward underexplored and informative regions of the state–action space. The demonstrated scalability of SOFTAE to such settings indicates that uncertainty-driven exploration remains effective even as observation complexity increases. p. 9

Possible Extensions

Future work could investigate extending active exploration to raw sensory inputs by combining epistemic uncertainty with learned latent state representations. Integrating multimodal sensing, such as RGB-D or tactile data, could improve robustness and applicability in less controlled settings. p. 0 Another promising direction is to integrate low-fidelity physical models as priors, which has been shown to yield orders-of-magnitude improvements in sample efficiency. p. 2

Materials and Methods

The authors study a general discrete-time nonlinear dynamical system of the form st+1 = f∗(st, at) + wt, where the system dynamics f∗ are unknown and they aim to learn them from data. They use probabilistic ensembles for uncertainty quantification in this work. p. 4 The core algorithm, SOFTAE, maximizes an intrinsic reward function based on the epistemic uncertainty about f∗ as advocated by Sukhija et al. Specifically, the objective is to approximate f∗ over the reachable state–action space. p. 1

Learning Uncertainty Aware Dynamics

They use an ensemble of neural networks to learn an uncertainty-aware state space model by approximating the posterior distribution with a set of L NN parameter particles. The mean and model uncertainty are estimated using Equations (8). p. 1 As data is collected, they expect the different particle initializations to converge to the same prediction and therefore their epistemic uncertainty to decrease. p. 9

Trajectory and Policy Optimization

The authors discuss two approaches for obtaining a closed-loop policy for data collection: Trajectory Optimization and Model-Predictive Control, which involves solving an open-loop action sequence optimization at each time step. Alternatively, they use Model-Based Policy Optimization, where they sample a batch of states and train the policy with Soft Actor-Critic (SAC) augmented with synthetic rollouts from the learned dynamics model. p. 2

Soft Robotic Simulated Environments and Tasks

The simulated environments include a soft continuum arm modeled as a Cosserat rod, an articulated fish with deformable skin in fluid, and a musculoskeletal leg actuated by electrohydraulic muscles. These systems present unique challenges from continuous elastic deformation to two-way fluid–structure interaction and hybrid muscle-driven actuation. p. 4 The state spaces for these environments vary significantly, ranging from 58D to 136D in the case of the continuum arm. p. 6

Experiment Details

The hyperparameters used for experiments are detailed in Tables S6 through S8. The SAC hyperparameters correspond to the model-based policy training experiments on the simulated soft continuum arm as reported in Figure S2. p. 0 The state and action spaces for all tested environments are decomposed in Tables S1 through S4. p. 1

Environment Details

Table S4 presents the real-world soft continuum SoPrA arm environment, which is tracked using a motion capture software and actuated by six differential pressure commands. The state space for the SoPrA arm includes position, linear velocity, orientation, and angular velocity of both rings relative to the base ring. p. 9

Table S5

Table S5 summarizes the reward definitions for all simulated environments described in Table 1 as well as for the real-world SoPrA platform. The task rewards are formulated as a function of position, linear velocity, or angular displacement depending on the environment and objective. p. 9

Table S6

Table S6 lists the dynamics model training hyperparameters for all soft robotic environments, covering exploration and task horizons, dynamics model network architecture and ensemble size. The hyperparameters for the iCEM optimizer are also reported in Table S7. p. 4

Table S7

Table S7 reports the optimizer settings used in this work, including the number of samples, horizon length, elite-set size, colored-noise exponent, number of particles, CEM iterations, and the fraction of elites reused.

Table S8

Table S8 lists the training hyperparameters used for the SAC agent in the model-based RL setting. The imagined rollout horizon H is specified in Table S8. p. 8 p. 8

Figure 1 Caption for Movie S1

Caption for Movie S1. Simulated soft continuum arm via Cosserat rods. p. 4 This video first presents the environment setup, including the state and action space decomposition (see Table S1), before demonstrating the downstream reaching tasks (i)–(ii). p. 6

Figure 2 Caption

Figure 2: Active exploration enables robust zero-shot task performance across diverse soft robotic systems. SOFTAE consistently produces task-aligned and physically plausible behaviors, while RANDOM and H-UCRL often fail to generalize to the hard or unseen scenarios. p. 7

Figure 3 Caption

Figure 3: Active exploration improves data coverage, model accuracy, and scales to high-dimensional dynamics. SOFTAE achieves the lowest error, indicating more globally accurate models. p. 8

Table S1 State and Action Space Decomposition of the Soft Continuum Arm

Table S1 outlines the state and action space decomposition for the simulated soft continuum arm, which is discretized into either 5 or 11 elements along its body, yielding 58D and 136D state spaces respectively.

Table S2 State and Action Space Decomposition of the Articulated Fish with Deformable Skin

Table S2 summarizes the state and action space for the articulated fish with deformable skin, as used in tasks (iii)–(vi) with a 15D state space and a 18D state space respectively.

Table S3 State and Action Space Decomposition for the Musculoskeletal Leg

Table S3 describes the musculoskeletal (MSK) leg environment, which is controlled via a combination of a DC motor at the boom joint and four voltage inputs to antagonistic electrohydraulic muscles.

Table S4 State and Action Space Decomposition for the Real-World Pneumatically Actuated Soft Arm

Table S4 presents the real-world soft continuum SoPrA arm environment, detailing the state space for both rings relative to the base ring along with previous step control commands.

Figure 4 Caption for Movie S4

Caption for Movie S4. Real-world pneumatically actuated SoPrA arm. p. 4 This video presents the real-world SoPrA environment and task performance and is the animated equivalent of Figure 4. p. 4

Table S5 Downstream Task Rewards

Table S5 summarizes the reward definitions for all simulated environments described in Table 1 as well as for the real-world SoPrA platform. The arm reach tasks employ a shaped long-tail reward function that maps the distance to target into the range (0, 1] where it is less than 5 × 10−3. p. 6

Table S6 Hyperparameters

Table S6 lists the dynamics model training hyperparameters for all soft robotic environments, covering exploration and task horizons, dynamics model network architecture and ensemble size. The learning rate and number of epochs are also reported in Table S6. p. 6

Table S7 iCEM Optimizer Hyperparameters

Table S7 reports the optimizer settings used in this work, including the number of samples, horizon length, elite-set size, colored-noise exponent, number of particles, CEM iterations, and the fraction of elites reused.

Table S8 SAC Algorithm Hyperparameters

Table S8 lists the training hyperparameters used for the SAC agent in the model-based RL setting. The imagined rollout horizon H is specified in Table S8.

Figure S1 Comparison of Different Uncertainty-Driven Active Exploration Strategies

Figure S1 compares SOFTAE with two additional active exploration baselines, MEAN-AE and PETS-AE, alongside RANDOM exploration and the task-specific model-based method H-UCRL. SOFTAE achieves the most consistent and highest returns across tasks. p. 7

Figure S2 Return Comparison of Policy Optimization in the Soft Continuum Arm Simulation

Figure S2 shows that SOFTAE+MBRL achieves performance comparable to task-specific baselines on the training task, but substantially outperforms them on unseen downstream tasks.

Table S11 Hyperparameters for iCEM Optimizer

Table S11 lists the hyperparameters used for the iCEM optimizer across different environments and tasks.

Table S25 Environment Details

Table S25 provides detailed descriptions of the state and action spaces for the three simulated soft robotic environments and one real-world setup. The state entries include element positions, velocities, orientations, and angular velocities, while the action space consists of torques applied at six equidistant points along the arm. p. 9

Table S4 Real-World SoPrA Arm Environment Details

Table S4 presents the real-world soft continuum SoPrA arm environment, detailing the state space for both rings relative to the base ring along with previous step control commands. The action entry corresponds to delta pressures applied to each of the six chambers. p. 9 p. 9

Table S1 Downstream Task Reward Definitions

Table S1 provides detailed descriptions of the state and action spaces for the simulated soft continuum arm, which is discretized into either 5 or 11 elements along its body, yielding 58D and 136D state spaces respectively. The action space consists of torques applied in the normal and binormal directions at six control points. p. 9

Table S2 State and Action Space Decomposition for the Articulated Fish with Deformable Skin

Table S2 summarizes the state and action space for the articulated fish with deformable skin, as used in tasks (iii)–(vi) with a 15D state space and an 18D state space respectively.

Table S4 State and Action Space Decomposition for the Real-World SoPrA Arm

Table S4 presents the real-world soft continuum SoPrA arm environment, detailing the state space for both rings relative to the base ring along with previous step control commands. The action entry corresponds to delta pressures applied to each of the six chambers.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Data acquisition strategy improvement: SOFTAE autonomously learns task-agnostic dynamics models by using optimism tailored to the unique challenges of soft robotics as an intrinsic motivation signal, shifting data collection away from passively following task-specific trajectories or random perturbations and instead focuses on maximizing information gain during training.

  2. Generalization capability: The resulting dynamics models enable superior zero-shot control on unseen tasks, allowing the model to be directly used in downstream motion planning across diverse morphologies and actuation regimes without retraining.

  3. Robustness to uncertainty: By targeting regions of high model uncertainty, the method improves data coverage and accuracy, leading to dynamics models that maintain robustness under sensing noise, actuation delays, and nonlinear material effects.

  4. Scalability for high-dimensional systems: SOFTAE demonstrates scalability to higher observation dimensionality by extracting features from more evenly spaced points (e.g., 136D state space), maintaining high performance on both reaching tasks despite the increased complexity.

  5. Real-world deployment capability: The framework is validated on physical hardware, showing that SOFTAE achieves the highest mean return and lowest tip position error compared to baselines across diverse real-world tasks, including handling sensor noise and actuation delays.

  6. Policy optimization flexibility: The system can utilize either trajectory optimization or model-based policy optimization, allowing for a closed-loop controller that is either simpler and less sensitive to bias (trajectory optimization) or more sample efficient (model-based policy optimization).

Sources

Related papers