Model Predictive Control for Dynamic Cloth Manipulation: Parameter Learning and Experimental Validation

arXiv:2209.05798 · cs.RO · Submitted 2022-09-13 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "Model Predictive Control for Dynamic Cloth Manipulation".

Dev: Here's the gist of the paper. Robotic cloth manipulation is addressed by applying a Model Predictive Control (MPC) strategy that uses a linear cloth model to indirectly control point configurations by acting on other parts of the object The designed controller finds the optimal control signals to attain the desired future target configuration This work proposes and validates a linear cloth model that allows solving the MPC-related optimization problem in real time, which is then tuned using Reinforcement Learning (RL) techniques The full control scheme was implemented and executed in a real robot, obtaining accurate tracking even in adverse conditions.

Rosa: Problem Formulation The main application of this paper is to control the movement of a piece of cloth so that certain parts of such cloth, the interest points, track a reference trajectory by controlling other grasped points In this proof of concept we do not include collision scenarios, leaving this aspect as further work.

Dev: The manipulation is done with two robotic arms by holding the piece of cloth in the air with two pinch grasps In our case we manipulate the cloth by varying the position (x, y, z coordinates) of the two upper corners, which are our grasped points u ∈ U ⊂ R 6 where U = u ≤ u u with u and u being the minimum and maximum control signals respectively.

Rosa: The reference trajectory is defined by y = rT ∈ Y ⊂ R 6 where Y = y ≤ y y with y and y being the minimum and maximum position values for each lower corner respectively.

Paper summary: Rosa: So to summarize this paper, they are focused on indirectly controlling specific points on a textile object by manipulating other parts of it.

Dev: They achieve this by using Model Predictive Control to predict future cloth states based on a simplified linear model, which allows the controller to calculate the best control signals in real time.

Rosa: The key claim is that they propose and validate this linear cloth model and tune its parameters using Reinforcement Learning techniques.

Dev: This whole process is designed so that the controller can find those optimal control signals to get the cloth into a desired future configuration.

Taro: It seems like the real significance here is not just making it work in simulation, but proving that you can achieve this kind of prediction and control using a model that actually resembles the real cloth dynamically.

Rosa: Exactly. They’re showing that you don't need to predict the entire high dimensional system of equations for every single node to get success.

Dev: The paper suggests that just predicting a few relevant points, like the lower corners and the grasped points, gives you most of the information needed for a task <ref:2209.05798#pg1>.

Taro: That moves it from being a purely theoretical simulation problem to something that has practical relevance for real-world robotic manipulation where things are constantly changing.

Rosa: It’s about showing that this MPC approach can be adapted to handle the dynamic relationships among all the points of the cloth, even if you have to use some prior knowledge of the cloth dynamics <ref:2209.05798#pg3>.

Dev: So, it sets up a framework where you combine system and environment behavioral prediction with prior knowledge of how the cloth moves.

Conclusion: Rosa: Looking at this paper, "Model Predictive Control for Dynamic Cloth Manipulation: Parameter Learning and Experimental Validation," it really seems like a proof of concept for a new way to handle cloth manipulation.

Dev: It's about using the MPC strategy with cloth dynamics prediction in real time, and what makes it feasible is that you don't need to predict the entire cloth mesh evolution accurately.

Rosa: The authors show that just tracking a few key points—like the lower corners and your grasped points—can give you most of what’s needed to succeed in a task <ref:2209.05798#pg1>.

Dev: The implication is that even with disturbances, the error analysis suggests that the errors are around five centimeters, and this isn't primarily due to the predictive controller itself.

Rosa: So, for someone just listening to the show, it means you can get a proof of concept for using MPC with cloth dynamics prediction in real time.

Dev: It points toward a methodology where you leverage a few relevant points instead of needing perfect knowledge of every single point’s movement <ref:2209.05798#pg1>.

Taro: It suggests that the system can be robust enough to handle some external disturbances, which is important when you consider real-world deployment.

Rosa: Right. It's about taking a complex object like cloth and making it controllable using a simplified model that works in practice.

Adria Luque, David Parent, ¡ a Colom ¡ e, Carlos Ocampo-Martinez, Carme Torras

Institut de Robotica i Inform ¡tica Industrial, CSIC-UPC, Barcelona, Spain · Universitat Politecnica de Catalunya - BarcelonaTECH, Barcelona TECH

cs.RO

Submitted: 2022-09-13

Updated: 2022-09-20

DOI: 10.1109/TCST.2024.3362514

Code: https://github.com/MiguelARD/cloth_point_cloud_segmentation

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 69/100

The gist: Robotic cloth manipulation is addressed by applying a Model Predictive Control (MPC) strategy that uses a linear cloth model to indirectly control point configurations by acting on other parts of the…

Key concepts

Model Predictive Control (MPC)
A control strategy that uses a mathematical model of the system to predict future behavior and calculate the best sequence of control actions to minimize a cost function over a future time horizon. It allows the robot to plan moves ahead instead of just reacting to current errors.
Linear Cloth Model
A simplified, mass-spring-damper representation used to mimic the physical dynamics of a piece of cloth. This model simplifies complex real-world cloth behavior into a set of equations that are easier for the MPC to solve quickly in real time.
Relative Entropy Policy Search (REPS)
A Reinforcement Learning technique used here to train the linear cloth model parameters. It works by penalizing errors in the positions of all nodes in the cloth mesh, but it gives more importance (higher weights) to keeping the lower corner nodes accurate, as those are the points being tracked.
Cost Function ($J_k$)
A mathematical function that measures how good a current control action is. It balances two main goals: minimizing the error between the cloth's actual position and the desired reference trajectory, and keeping control signals small to ensure smooth, stable movement.

Terminology

Summary

Robotic cloth manipulation is addressed by applying a Model Predictive Control (MPC) strategy that uses a linear cloth model to indirectly control point configurations by acting on other parts of the object The designed controller finds the optimal control signals to attain the desired future target configuration This work proposes and validates a linear cloth model that allows solving the MPC-related optimization problem in real time, which is then tuned using Reinforcement Learning (RL) techniques The full control scheme was implemented and executed in a real robot, obtaining accurate tracking even in adverse conditions. p. 0

Where the paper says it

This paper addresses the problem of indirectly controlling the configuration of certain points of a textile object, by applying actions on other parts of the object through the use of a Model Predictive Control (MPC) strategy, which also allows to foresee the behavior of indirectly controlled points. The designed controller finds the optimal control signals to attain the desired future target configuration.

Problem Formulation

The main application of this paper is to control the movement of a piece of cloth so that certain parts of such cloth, the interest points, track a reference trajectory by controlling other grasped points In this proof of concept we do not include collision scenarios, leaving this aspect as further work. The manipulation is done with two robotic arms by holding the piece of cloth in the air with two pinch grasps In our case we manipulate the cloth by varying the position (x, y, z coordinates) of the two upper corners, which are our grasped points u ∈ U ⊂ R 6 where U = u ≤ u u with u and u being the minimum and maximum control signals respectively. p. 1

Where the paper says it

For this tracking endeavor, we consider that the manipulation is done with two robotic arms by holding the piece of cloth in the air with two pinch grasps. Hence, in our case we manipulate the cloth by varying the position (x, y, z coordinates) of the two upper corners, which are our grasped points u ∈ U ⊂ R 6 (see Fig.

The reference trajectory is defined by y = rT ∈ Y ⊂ R 6 where Y = y ≤ y y with y and y being the minimum and maximum position values for each lower corner respectively. p. 1
Where the paper says it

In our case, defining the trajectory of both lower corners y = {yr, yl} T ∈ Y ⊂ R 6 is enough to generate a reference trajectory to track, namely r = {rr, rl} T ∈ Y ⊂ R 6 (see Fig. 2), where Y = {y ∈ R 6 : y ≤ y ≤ y}, with y and y being the minimum and maximum position values for each lower

Model Development

The proposed Control-Oriented Model (COM) is a mass-spring-damper system, which should be able to reproduce those system dynamics to be controlled afterwards while maintaining highest degree of accuracy with respect to the real cloth. The system can be represented as a graph of N nodes belonging to a set I = I = 1, 2, …, N and a set of edges E that represent the springs connecting those nodes. p. 2

Where the paper says it

3: Mass-spring-damper system with structural springs (N = n 2 = 16) The system in Figure 3 can be represented as a graph of N nodes belonging to a set I = {1, 2, . , N} and a set of edges E that represent the springs connecting those nodes.

The resulting force of the springs S ik is computed as S ik = X Ni-K kd j k k - l j 0 kd j k d j k, where K is the spring stiffness and l j 0 is the initial length of the spring connecting that j neighbor with the i-th particle. p. 2
Where the paper says it

In our case, we only use structural springs –which yield a sufficiently proper results and save computational costs–, therefore, ρ = 4 for the interior points of the mesh. For each particle i ∈ I, we can compute the resulting force of the springs S i k in time step k ∈ Z≥0 as S i k = X Ni −K kd j k k − l j 0 kd j k k d j k , where K is the spring stiffness, d j k ∈ R 3 is the vector pointing from the neighbor j ∈ Ni to the particle i ∈ I and l j 0 ∈ R>0 is the initial length of the spring connecting that j neighbor with the i-th particle.

The system state-space realization is given by xk+1 = Axk + Buk + fct, where x ∈ X ⊂ R 6N is the vector of the cloth states (position p and velocity v in x, y, z of nodes i ∈ I). p. 3
Where the paper says it

, S i k ≈ X Ni    −kx(p i x,k − p j x,k − l j 0x ) −ky(p i y,k − p j y,k − l j 0y ) −kz(p i z,k − p j z,k − l j 0z )    . With this transformation, we have a linear system that can be expressed through a state-space realization as xk+1 = Axk + Buk + fct, yk = Cxk, (1) where x ∈ X ⊂ R 6N is the vector of the cloth states (position p and velocity v in x, y, z of nodes i ∈ I), u ∈ U ⊂ R 6 is the control input vector, y ∈ Y ⊂ R 6 is the output vector (positions of the lower corners) and fct ∈ F ⊂ R 6N is the vector of constant forces applied to each node: gravity and natural spring length force.

Control Strategy Design

The final multi-objective cost function at each time step k ∈ Z≥0 is Jk = H Xp-1 i=0 kyk+i+1k − rk+i+1k squared Q + k∆uk+ikk squared R, where the notation (k + ik) refers to the prediction at time step (k+i) based on measurements made at time step k. The optimization problem formulation is min us k Jk, s. p. 4

Where the paper says it

3) Optimization problem formulation: Based on the receding horizon strategy [5] and considering a fixed prediction horizon Hp, the goal is to obtain the input sequences that minimize the tracking errors of the two lower corners while satisfying all constraints. The controller is based on the solution of the following discrete-time open-loop optimization problem (OOP): min us k Jk, s.

t. xk+i+1k = Axk+ik + Buk+ikk + fct yk+ik = Cxk+iK x l k = x uk-1K = uk-1. p. 3
Where the paper says it

, S i k ≈ X Ni    −kx(p i x,k − p j x,k − l j 0x ) −ky(p i y,k − p j y,k − l j 0y ) −kz(p i z,k − p j z,k − l j 0z )    . With this transformation, we have a linear system that can be expressed through a state-space realization as xk+1 = Axk + Buk + fct, yk = Cxk, (1) where x ∈ X ⊂ R 6N is the vector of the cloth states (position p and velocity v in x, y, z of nodes i ∈ I), u ∈ U ⊂ R 6 is the control input vector, y ∈ Y ⊂ R 6 is the output vector (positions of the lower corners) and fct ∈ F ⊂ R 6N is the vector of constant forces applied to each node: gravity and natural spring length force.

An adaptive tuning of the Q matrix is proposed where Qa,k = diag βk, with an term included to avoid numerical problems when the corner is exactly at the reference. p. 4
Where the paper says it

In each time step, we compute the distance vector between the current position and that of the reference at the end of the prediction horizon, finding the main direction β in which the lower corners move along Hp. By normalizing this vector, the final adaptive weighting matrix Qa is computed as βk = |rk+Hp − yk|k| krk+Hp − yk|kk + , Qa,k = diag βk , (5) where an term has been included to avoid numerical problems when the corner is exactly at the reference.

Experimental Validation and Results

The linear model presented in (1) has seven parameters to tune in order for the linear model to follow the same behavior as a real cloth. The relative entropy policy search (REPS) algorithm was used [28] penalizing the squared errors in node positions for the entire mesh, but with higher weights for the lower corner nodes. p. 8

Where the paper says it

After being filtered and regularized in time, these evolutions were used to find the parameters of the linear model via Reinforcement Learning (RL). Concretely, the Relative Entropy Policy Search (REPS) algorithm was used [28], penalizing the squared errors in node positions for the entire mesh, but with higher weights for the lower corner nodes, as they will be the output to be tracked.

The final tracking error is approximately the same regardless of the chosen weight structure (less than 0. p. 9
Where the paper says it

Knowing how time measurements are sensitive to memory and CPU usage, and obtaining roughly the same results with both reward functions, the second one, with TOV, was discarded for the following experiments. The final tracking error is approximately the same regardless of the chosen weight structure (less than 0.

5% difference between best and worst). p. 9
Where the paper says it

The final tracking error is approximately the same regardless of the chosen weight structure (less than 0. 5% difference between best and worst), and the minimum errors are actually obtained with the first and simplest option.

p. 9
Where the paper says it

The final tracking error is approximately the same regardless of the chosen weight structure (less than 0. 5% difference between best and worst), and the minimum errors are actually obtained with the first and simplest option.

The final RMSE of the experiment is of 4. p. 6
Where the paper says it

The nonlinear SOM substitutes the real cloth in simulation, and the linear B-SOM acts as a fast backup while the real feedback is processed, which in the real setup can take several time steps compute the norm of each error and the final average, i. , e = RMSE(y − r), (8a) KPI1 = 1 2 ke r k + ke l k , (8b) KPI2 = ¯τ.

1 cm, but the error is not constant or distributed evenly, as it reaches maximums of 4. p. 2
Where the paper says it

For each particle i ∈ I, we can compute the resulting force of the springs S i k in time step k ∈ Z≥0 as S i k = X Ni −K kd j k k − l j 0 kd j k k d j k , where K is the spring stiffness, d j k ∈ R 3 is the vector pointing from the neighbor j ∈ Ni to the particle i ∈ I and l j 0 ∈ R>0 is the initial length of the spring connecting that j neighbor with the i-th particle. The resulting force of the dampers Di k is calculated as Di k = P Ni −b(v i k − v j k ), where b is the damping constant and vk ∈ R 3 is the linear velocity of a particle.

4, 5.7 and 5.8 cm in X, Y and Z respectively. p. 1
Where the paper says it

Hence, in our case we manipulate the cloth by varying the position (x, y, z coordinates) of the two upper corners, which are our grasped points u ∈ U ⊂ R 6 (see Fig. 2), where U = {u ∈ R 6 : u ≤ u ≤ u}, with u and u being the minimum and maximum control signals, respectively.

Conclusion

The outcome of this paper can serve as a proof of concept of a new methodology for robotic cloth manipulation: using MPC with cloth dynamics prediction in real-time. A key aspect that allows for such real-time capability is the fact that, unlike other approaches [12], we do not need to predict an accurate evolution for the entire cloth mesh, but rather few relevant points (e.g., lower corners plus the grasped points) can provide most of the information needed to achieve success in a task. p. 2

Where the paper says it

Controlling the position of the lower corners by only moving the upper corners (the grasped ones) is not a simple task, specially when they are far from each other. Dynamic relationships among all the points of the piece of cloth are mostly nonlinear and depend on both the position and velocity of the other points of the cloth.

The paper suggests that even with disturbances, the errors (RMSE) are around the 5 cm mark, and an error analysis shows its principal source is not the Predictive Controller. p. 0
Where the paper says it

After obtaining accurate tracking results in simulation, the full control scheme was implemented and executed in a real robot, obtaining accurate tracking even in adverse conditions. While total observed errors reach the 5 cm mark, for a 30×30 cm cloth, an analysis shows the MPC contributes less than 30% to that value.

Pruning redundant reasoning tokens

The paper addresses the problem of indirectly controlling the configuration of certain points of a textile object by applying actions on other parts of the object through the use of a Model Predictive Control (MPC) strategy. The explored scenario considers tracking a reference trajectory with the lower corners of a square piece of cloth by grasping its upper corners. p. 0

Where the paper says it

The designed controller finds the optimal control signals to attain the desired future target configuration. The explored scenario in this paper considers tracking a reference trajectory with the lower corners of a square piece of cloth by grasping its upper corners.

Reinforcement Learning (RL) techniques are used to learn the optimal parameters of the proposed cloth model and also to tune the resulting MPC. p. 0
Where the paper says it

To do so, we propose and validate a linear cloth model that allows solving the MPC-related optimization problem in real time. Reinforcement Learning (RL) techniques are used to learn the optimal parameters of the proposed cloth model and also to tune the resulting MPC.

How the training signal is built

The relative entropy policy search (REPS) algorithm was used [28] penalizing the squared errors in node positions for the entire mesh, but with higher weights for the lower corner nodes, as they will be the output to be tracked. The learning experiments were conducted to learn the model parameters for square meshes of sizes n = 4 and 7, and for time steps of Ts = 10, 15, 20 and 25 ms. p. 8

Where the paper says it

Concretely, the Relative Entropy Policy Search (REPS) algorithm was used [28], penalizing the squared errors in node positions for the entire mesh, but with higher weights for the lower corner nodes, as they will be the output to be tracked. Learning experiments were conducted to learn the model parameters for square meshes of sizes n = 4 and 7, and for time steps of Ts = 10, 15, 20 and 25 ms, obtaining results for all eight combinations.

Where the gains come from

The final tracking error is approximately the same regardless of the chosen weight structure (less than 0.5% difference between best and worst). The idea is getting as close as possible to the limit without crossing it for any case, leading to setting a unique tuning to guarantee the best performance in any given situation. p. 9

Where the paper says it

This led us to set a unique tuning to guarantee the best performance in any given situation. The idea is getting as close as possible to the limit without crossing it for any case.

A possible combination that works safely for all considered trajectories is R/Q = 0. p. 9
Where the paper says it

The idea is getting as close as possible to the limit without crossing it for any case. A possible combination that works safely for all considered trajectories is R/Q = 0.

2 (e.g., q = 1, r = 0.2). The final tracking performance shows that the MPC output follows the reference trajectory closely in situations of constant velocity, but it tends to smooth out changes in direction.

The paper is developed in the context of the project CLOTHILDE (”CLOTH manIpulation Learning from DEmonstrations”) which has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programmee. All authors are with the Institut de Robotica i Inform atica Industrial, CSIC-UPC, Barcelona, Spain. p. 0

Where the paper says it

Highly deformable objects represent a major challenge due to their physical properties: their shape and appearance can continuously vary during This work was developed in the context of the project CLOTHILDE (”CLOTH manIpulation Learning from DEmonstrations”), which has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programmee (Advanced Grant agreement No 741930). All authors are with the Institut de Robotica i Inform ` atica Industrial, ` CSIC-UPC, Barcelona, Spain.

The work was developed in the context of the project CLOTHILDE (”CLOTH manIpulation Learning from DEmonstrations”) which has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programmee. p. 0
Where the paper says it

A simple task such as taking a cloth garment from a rack and putting it in a box for shipping is still an open problem which the industry is trying to find a solution for. Highly deformable objects represent a major challenge due to their physical properties: their shape and appearance can continuously vary during This work was developed in the context of the project CLOTHILDE (”CLOTH manIpulation Learning from DEmonstrations”), which has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programmee (Advanced Grant agreement No 741930).

The authors are with the Institut de Robotica i Inform atica Industrial, CSIC-UPC, Barcelona, Spain

The paper is developed in the context of the project CLOTHILDE (”CLOTH manIpulation Learning from DEmonstrations”) which has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programmee. All authors are with the Institut de Robotica i Inform atica Industrial, CSIC-UPC, Barcelona, Spain

The paper is developed in the context of the project CLOTHILDE (”CLOTH manIpulation Learning from DEmonstrations”) which has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programmee.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Model Predictive Control for Dynamic Cloth Manipulation: The system can indirectly control cloth points by applying actions on other parts of the object through a Model Predictive Control strategy to foresee future behaviors, finding optimal control signals to attain desired future target configurations.

  2. Linear Cloth Model Design: A linear cloth model based on a mass-spring-damper system can be used for real-time MPC, which is faster than highly nonlinear models but accurate enough for tracking lower corners, and this model can be learned using Reinforcement Learning techniques.

  3. Model Parameter Learning: Reinforcement Learning techniques, such as Relative Entropy Policy Search (REPS), can be used to learn the optimal parameters of the proposed cloth model and to tune the resulting MPC controller structure and weights automatically.

  4. Adaptive Weight Tuning: An adaptive tuning of the Q matrix is proposed where an adaptive weighting matrix Qa is computed based on the distance vector between current position and reference at the end of a prediction horizon, helping to minimize tracking errors dynamically.

  5. Real-Time Implementation with Backup Model: The closed-loop control scheme can be implemented in real time by using a linear Backup Simulation Oriented Model (B-SOM) to simulate the cloth with control signals on each time step, providing updated initial conditions even when real feedback is delayed or unreliable.

Abstract

Robotic cloth manipulation is a relevant challenging problem for autonomous robotic systems. Highly deformable objects as textile items can adopt multiple configurations and shapes during their manipulation. Hence, robots should not only understand the current cloth configuration but also be able to predict the future possible behaviors of the cloth. This paper addresses the problem of indirectly controlling the configuration of certain points of a textile object, by applying actions on other parts of the object through the use of a Model Predictive Control (MPC) strategy, which also allows to foresee the behavior of indirectly controlled points. The designed controller finds the optimal control signals to attain the desired future target configuration. The explored scenario in this paper considers tracking a reference trajectory with the lower corners of a square piece of cloth by grasping its upper corners. To do so, we propose and validate a linear cloth model that allows solving the MPC-related optimization problem in real time. Reinforcement Learning (RL) techniques are used to learn the optimal parameters of the proposed cloth model and also to tune the resulting MPC. After obtaining accurate tracking results in simulation, the full control scheme was implemented and executed in a real robot, obtaining accurate tracking even in adverse conditions. While total observed errors reach the 5 cm mark, for a 30x30 cm cloth, an analysis shows the MPC contributes less than 30% to that value.

Related papers