Model Predictive Control for Dynamic Cloth Manipulation: Parameter Learning and Experimental Validation
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: We're looking at the paper: "Model Predictive Control for Dynamic Cloth Manipulation".
Dev: Here's the gist of the paper. Robotic cloth manipulation is addressed by applying a Model Predictive Control (MPC) strategy that uses a linear cloth model to indirectly control point configurations by acting on other parts of the object The designed controller finds the optimal control signals to attain the desired future target configuration This work proposes and validates a linear cloth model that allows solving the MPC-related optimization problem in real time, which is then tuned using Reinforcement Learning (RL) techniques The full control scheme was implemented and executed in a real robot, obtaining accurate tracking even in adverse conditions.
Rosa: Problem Formulation The main application of this paper is to control the movement of a piece of cloth so that certain parts of such cloth, the interest points, track a reference trajectory by controlling other grasped points In this proof of concept we do not include collision scenarios, leaving this aspect as further work.
Dev: The manipulation is done with two robotic arms by holding the piece of cloth in the air with two pinch grasps In our case we manipulate the cloth by varying the position (x, y, z coordinates) of the two upper corners, which are our grasped points u ∈ U ⊂ R 6 where U = u ≤ u u with u and u being the minimum and maximum control signals respectively.
Rosa: The reference trajectory is defined by y = rT ∈ Y ⊂ R 6 where Y = y ≤ y y with y and y being the minimum and maximum position values for each lower corner respectively.
Paper summary: Rosa: So to summarize this paper, they are focused on indirectly controlling specific points on a textile object by manipulating other parts of it.
Dev: They achieve this by using Model Predictive Control to predict future cloth states based on a simplified linear model, which allows the controller to calculate the best control signals in real time.
Rosa: The key claim is that they propose and validate this linear cloth model and tune its parameters using Reinforcement Learning techniques.
Dev: This whole process is designed so that the controller can find those optimal control signals to get the cloth into a desired future configuration.
Taro: It seems like the real significance here is not just making it work in simulation, but proving that you can achieve this kind of prediction and control using a model that actually resembles the real cloth dynamically.
Rosa: Exactly. They’re showing that you don't need to predict the entire high dimensional system of equations for every single node to get success.
Dev: The paper suggests that just predicting a few relevant points, like the lower corners and the grasped points, gives you most of the information needed for a task <ref:2209.05798#pg1>.
Taro: That moves it from being a purely theoretical simulation problem to something that has practical relevance for real-world robotic manipulation where things are constantly changing.
Rosa: It’s about showing that this MPC approach can be adapted to handle the dynamic relationships among all the points of the cloth, even if you have to use some prior knowledge of the cloth dynamics <ref:2209.05798#pg3>.
Dev: So, it sets up a framework where you combine system and environment behavioral prediction with prior knowledge of how the cloth moves.
Conclusion: Rosa: Looking at this paper, "Model Predictive Control for Dynamic Cloth Manipulation: Parameter Learning and Experimental Validation," it really seems like a proof of concept for a new way to handle cloth manipulation.
Dev: It's about using the MPC strategy with cloth dynamics prediction in real time, and what makes it feasible is that you don't need to predict the entire cloth mesh evolution accurately.
Rosa: The authors show that just tracking a few key points—like the lower corners and your grasped points—can give you most of what’s needed to succeed in a task <ref:2209.05798#pg1>.
Dev: The implication is that even with disturbances, the error analysis suggests that the errors are around five centimeters, and this isn't primarily due to the predictive controller itself.
Rosa: So, for someone just listening to the show, it means you can get a proof of concept for using MPC with cloth dynamics prediction in real time.
Dev: It points toward a methodology where you leverage a few relevant points instead of needing perfect knowledge of every single point’s movement <ref:2209.05798#pg1>.
Taro: It suggests that the system can be robust enough to handle some external disturbances, which is important when you consider real-world deployment.
Rosa: Right. It's about taking a complex object like cloth and making it controllable using a simplified model that works in practice.
Adria Luque, David Parent, ¡ a Colom ¡ e, Carlos Ocampo-Martinez, Carme Torras
Institut de Robotica i Inform ¡tica Industrial, CSIC-UPC, Barcelona, Spain · Universitat Politecnica de Catalunya - BarcelonaTECH, Barcelona TECH
cs.RO
Submitted: 2022-09-13
Updated: 2022-09-20
DOI: 10.1109/TCST.2024.3362514
Code: https://github.com/MiguelARD/cloth_point_cloud_segmentation
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 69/100
The gist: Robotic cloth manipulation is addressed by applying a Model Predictive Control (MPC) strategy that uses a linear cloth model to indirectly control point configurations by acting on other parts of the…
Key concepts
- Model Predictive Control (MPC)
- A control strategy that uses a mathematical model of the system to predict future behavior and calculate the best sequence of control actions to minimize a cost function over a future time horizon. It allows the robot to plan moves ahead instead of just reacting to current errors.
- Linear Cloth Model
- A simplified, mass-spring-damper representation used to mimic the physical dynamics of a piece of cloth. This model simplifies complex real-world cloth behavior into a set of equations that are easier for the MPC to solve quickly in real time.
- Relative Entropy Policy Search (REPS)
- A Reinforcement Learning technique used here to train the linear cloth model parameters. It works by penalizing errors in the positions of all nodes in the cloth mesh, but it gives more importance (higher weights) to keeping the lower corner nodes accurate, as those are the points being tracked.
- Cost Function ($J_k$)
- A mathematical function that measures how good a current control action is. It balances two main goals: minimizing the error between the cloth's actual position and the desired reference trajectory, and keeping control signals small to ensure smooth, stable movement.
Terminology
Summary
Robotic cloth manipulation is addressed by applying a Model Predictive Control (MPC) strategy that uses a linear cloth model to indirectly control point configurations by acting on other parts of the object The designed controller finds the optimal control signals to attain the desired future target configuration This work proposes and validates a linear cloth model that allows solving the MPC-related optimization problem in real time, which is then tuned using Reinforcement Learning (RL) techniques The full control scheme was implemented and executed in a real robot, obtaining accurate tracking even in adverse conditions. p. 0 This paper addresses the problem of indirectly controlling the
configuration of certain points of a textile object, by applying
actions on other parts of the object through the use of a
Model Predictive Control (MPC) strategy, which also allows
to foresee the behavior of indirectly controlled points. The
designed controller finds the optimal control signals to attain
the desired future target configuration.Where the paper says it
Problem Formulation
The main application of this paper is to control the movement of a piece of cloth so that certain parts of such cloth, the interest points, track a reference trajectory by controlling other grasped points In this proof of concept we do not include collision scenarios, leaving this aspect as further work. The manipulation is done with two robotic arms by holding the piece of cloth in the air with two pinch grasps In our case we manipulate the cloth by varying the position (x, y, z coordinates) of the two upper corners, which are our grasped points u ∈ U ⊂ R 6 where U = u ≤ u u with u and u being the minimum and maximum control signals respectively. p. 1 For this tracking endeavor, we consider
that the manipulation is done with two robotic arms by
holding the piece of cloth in the air with two pinch grasps. Hence, in our case we manipulate the cloth by varying the
position (x, y, z coordinates) of the two upper corners, which
are our grasped points u ∈ U ⊂ R
6
(see Fig. In our case, defining the
trajectory of both lower corners y = {yr, yl}
T ∈ Y ⊂
R
6
is enough to generate a reference trajectory to track,
namely r = {rr, rl}
T ∈ Y ⊂ R
6
(see Fig. 2), where
Y = {y ∈ R
6
: y ≤ y ≤ y}, with y and y being
the minimum and maximum position values for each lowerWhere the paper says it
Where the paper says it
Model Development
The proposed Control-Oriented Model (COM) is a mass-spring-damper system, which should be able to reproduce those system dynamics to be controlled afterwards while maintaining highest degree of accuracy with respect to the real cloth. The system can be represented as a graph of N nodes belonging to a set I = I = 1, 2, …, N and a set of edges E that represent the springs connecting those nodes. p. 2 3: Mass-spring-damper system with structural springs (N = n
2 = 16)
The system in Figure 3 can be represented as a graph
of N nodes belonging to a set I = {1, 2, . , N} and a
set of edges E that represent the springs connecting those
nodes. In our case, we only use structural
springs –which yield a sufficiently proper results and save
computational costs–, therefore, ρ = 4 for the interior points
of the mesh. For each particle i ∈ I, we can compute the
resulting force of the springs S
i
k
in time step k ∈ Z≥0 as
S
i
k =
X
Ni
−K
kd
j
k
k − l
j
0
kd
j
k
k
d
j
k
,
where K is the spring stiffness, d
j
k ∈ R
3
is the vector
pointing from the neighbor j ∈ Ni
to the particle i ∈ I
and l
j
0 ∈ R>0 is the initial length of the spring connecting
that j neighbor with the i-th particle. ,
S
i
k ≈
X
Ni
−kx(p
i
x,k − p
j
x,k − l
j
0x
)
−ky(p
i
y,k − p
j
y,k − l
j
0y
)
−kz(p
i
z,k − p
j
z,k − l
j
0z
)
. With this transformation, we have a linear system that can
be expressed through a state-space realization as
xk+1 = Axk + Buk + fct,
yk = Cxk,
(1)
where x ∈ X ⊂ R
6N is the vector of the cloth states (position
p and velocity v in x, y, z of nodes i ∈ I), u ∈ U ⊂ R
6
is
the control input vector, y ∈ Y ⊂ R
6
is the output vector
(positions of the lower corners) and fct ∈ F ⊂ R
6N is
the vector of constant forces applied to each node: gravity
and natural spring length force.Where the paper says it
Where the paper says it
Where the paper says it
Control Strategy Design
The final multi-objective cost function at each time step k ∈ Z≥0 is Jk = H Xp-1 i=0 kyk+i+1k − rk+i+1k squared Q + k∆uk+ikk squared R, where the notation (k + ik) refers to the prediction at time step (k+i) based on measurements made at time step k. The optimization problem formulation is min us k Jk, s. p. 4 3) Optimization problem formulation: Based on the receding horizon strategy [5] and considering a fixed prediction
horizon Hp, the goal is to obtain the input sequences that
minimize the tracking errors of the two lower corners while
satisfying all constraints. The controller is based on the solution of the following discrete-time open-loop optimization
problem (OOP):
min
us
k
Jk,
s. ,
S
i
k ≈
X
Ni
−kx(p
i
x,k − p
j
x,k − l
j
0x
)
−ky(p
i
y,k − p
j
y,k − l
j
0y
)
−kz(p
i
z,k − p
j
z,k − l
j
0z
)
. With this transformation, we have a linear system that can
be expressed through a state-space realization as
xk+1 = Axk + Buk + fct,
yk = Cxk,
(1)
where x ∈ X ⊂ R
6N is the vector of the cloth states (position
p and velocity v in x, y, z of nodes i ∈ I), u ∈ U ⊂ R
6
is
the control input vector, y ∈ Y ⊂ R
6
is the output vector
(positions of the lower corners) and fct ∈ F ⊂ R
6N is
the vector of constant forces applied to each node: gravity
and natural spring length force. In each
time step, we compute the distance vector between the
current position and that of the reference at the end of the
prediction horizon, finding the main direction β in which the
lower corners move along Hp. By normalizing this vector,
the final adaptive weighting matrix Qa is computed as
βk =
|rk+Hp − yk|k|
krk+Hp − yk|kk +
, Qa,k = diag
βk
, (5)
where an term has been included to avoid numerical
problems when the corner is exactly at the reference.Where the paper says it
Where the paper says it
Where the paper says it
Experimental Validation and Results
The linear model presented in (1) has seven parameters to tune in order for the linear model to follow the same behavior as a real cloth. The relative entropy policy search (REPS) algorithm was used [28] penalizing the squared errors in node positions for the entire mesh, but with higher weights for the lower corner nodes. p. 8 After being
filtered and regularized in time, these evolutions were used
to find the parameters of the linear model via Reinforcement Learning (RL). Concretely, the Relative Entropy Policy
Search (REPS) algorithm was used [28], penalizing the
squared errors in node positions for the entire mesh, but with
higher weights for the lower corner nodes, as they will be
the output to be tracked. Knowing how time
measurements are sensitive to memory and CPU usage,
and obtaining roughly the same results with both reward
functions, the second one, with TOV, was discarded for the
following experiments. The final tracking error is approximately the same regardless of the chosen weight structure (less than 0. The final tracking error is approximately the same regardless of the chosen weight structure (less than 0. 5% difference
between best and worst), and the minimum errors are actually
obtained with the first and simplest option. The final tracking error is approximately the same regardless of the chosen weight structure (less than 0. 5% difference
between best and worst), and the minimum errors are actually
obtained with the first and simplest option. The nonlinear SOM substitutes the real cloth in simulation, and the linear B-SOM acts as a fast backup while the real
feedback is processed, which in the real setup can take several time steps
compute the norm of each error and the final average, i. ,
e = RMSE(y − r), (8a)
KPI1 =
1
2
ke
r
k + ke
l
k
, (8b)
KPI2 = ¯τ. For each particle i ∈ I, we can compute the
resulting force of the springs S
i
k
in time step k ∈ Z≥0 as
S
i
k =
X
Ni
−K
kd
j
k
k − l
j
0
kd
j
k
k
d
j
k
,
where K is the spring stiffness, d
j
k ∈ R
3
is the vector
pointing from the neighbor j ∈ Ni
to the particle i ∈ I
and l
j
0 ∈ R>0 is the initial length of the spring connecting
that j neighbor with the i-th particle. The resulting force of
the dampers Di
k
is calculated as Di
k =
P
Ni
−b(v
i
k − v
j
k
),
where b is the damping constant and vk ∈ R
3
is the linear
velocity of a particle. Hence, in our case we manipulate the cloth by varying the
position (x, y, z coordinates) of the two upper corners, which
are our grasped points u ∈ U ⊂ R
6
(see Fig. 2), where
U = {u ∈ R
6
: u ≤ u ≤ u}, with u and u being the
minimum and maximum control signals, respectively.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Conclusion
The outcome of this paper can serve as a proof of concept of a new methodology for robotic cloth manipulation: using MPC with cloth dynamics prediction in real-time. A key aspect that allows for such real-time capability is the fact that, unlike other approaches [12], we do not need to predict an accurate evolution for the entire cloth mesh, but rather few relevant points (e.g., lower corners plus the grasped points) can provide most of the information needed to achieve success in a task. p. 2 Controlling the position of the lower
corners by only moving the upper corners (the grasped ones)
is not a simple task, specially when they are far from each
other. Dynamic relationships among all the points of the
piece of cloth are mostly nonlinear and depend on both the
position and velocity of the other points of the cloth. After obtaining accurate tracking results in simulation,
the full control scheme was implemented and executed in a real
robot, obtaining accurate tracking even in adverse conditions. While total observed errors reach the 5 cm mark, for a 30×30
cm cloth, an analysis shows the MPC contributes less than 30%
to that value.Where the paper says it
Where the paper says it
Pruning redundant reasoning tokens
The paper addresses the problem of indirectly controlling the configuration of certain points of a textile object by applying actions on other parts of the object through the use of a Model Predictive Control (MPC) strategy. The explored scenario considers tracking a reference trajectory with the lower corners of a square piece of cloth by grasping its upper corners. p. 0 The
designed controller finds the optimal control signals to attain
the desired future target configuration. The explored scenario in this paper considers tracking a
reference trajectory with the lower corners of a square piece
of cloth by grasping its upper corners. To do so, we propose
and validate a linear cloth model that allows solving the
MPC-related optimization problem in real time. Reinforcement
Learning (RL) techniques are used to learn the optimal parameters of the proposed cloth model and also to tune the resulting
MPC.Where the paper says it
Where the paper says it
How the training signal is built
The relative entropy policy search (REPS) algorithm was used [28] penalizing the squared errors in node positions for the entire mesh, but with higher weights for the lower corner nodes, as they will be the output to be tracked. The learning experiments were conducted to learn the model parameters for square meshes of sizes n = 4 and 7, and for time steps of Ts = 10, 15, 20 and 25 ms. p. 8 Concretely, the Relative Entropy Policy
Search (REPS) algorithm was used [28], penalizing the
squared errors in node positions for the entire mesh, but with
higher weights for the lower corner nodes, as they will be
the output to be tracked. Learning experiments were conducted to learn the model
parameters for square meshes of sizes n = 4 and 7, and for
time steps of Ts = 10, 15, 20 and 25 ms, obtaining results
for all eight combinations.Where the paper says it
Where the gains come from
The final tracking error is approximately the same regardless of the chosen weight structure (less than 0.5% difference between best and worst). The idea is getting as close as possible to the limit without crossing it for any case, leading to setting a unique tuning to guarantee the best performance in any given situation. p. 9 This led us to set a unique tuning to guarantee the best
performance in any given situation. The idea is getting as
close as possible to the limit without crossing it for any case. The idea is getting as
close as possible to the limit without crossing it for any case. A possible combination that works safely for all considered
trajectories is R/Q = 0.Where the paper says it
Where the paper says it
The paper is developed in the context of the project CLOTHILDE (”CLOTH manIpulation Learning from DEmonstrations”) which has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programmee. All authors are with the Institut de Robotica i Inform atica Industrial, CSIC-UPC, Barcelona, Spain. p. 0 Highly deformable objects
represent a major challenge due to their physical properties:
their shape and appearance can continuously vary during
This work was developed in the context of the project CLOTHILDE
(”CLOTH manIpulation Learning from DEmonstrations”), which has received funding from the European Research Council (ERC) under the
European Union’s Horizon 2020 research and innovation programmee
(Advanced Grant agreement No 741930). All authors are with the Institut de Robotica i Inform ` atica Industrial, `
CSIC-UPC, Barcelona, Spain. A simple task
such as taking a cloth garment from a rack and putting it in a
box for shipping is still an open problem which the industry
is trying to find a solution for. Highly deformable objects
represent a major challenge due to their physical properties:
their shape and appearance can continuously vary during
This work was developed in the context of the project CLOTHILDE
(”CLOTH manIpulation Learning from DEmonstrations”), which has received funding from the European Research Council (ERC) under the
European Union’s Horizon 2020 research and innovation programmee
(Advanced Grant agreement No 741930).Where the paper says it
Where the paper says it
The paper is developed in the context of the project CLOTHILDE (”CLOTH manIpulation Learning from DEmonstrations”) which has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programmee. All authors are with the Institut de Robotica i Inform atica Industrial, CSIC-UPC, Barcelona, Spain
The paper is developed in the context of the project CLOTHILDE (”CLOTH manIpulation Learning from DEmonstrations”) which has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programmee.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Model Predictive Control for Dynamic Cloth Manipulation: The system can indirectly control cloth points by applying actions on other parts of the object through a Model Predictive Control strategy to foresee future behaviors, finding optimal control signals to attain desired future target configurations.
-
Linear Cloth Model Design: A linear cloth model based on a mass-spring-damper system can be used for real-time MPC, which is faster than highly nonlinear models but accurate enough for tracking lower corners, and this model can be learned using Reinforcement Learning techniques.
-
Model Parameter Learning: Reinforcement Learning techniques, such as Relative Entropy Policy Search (REPS), can be used to learn the optimal parameters of the proposed cloth model and to tune the resulting MPC controller structure and weights automatically.
-
Adaptive Weight Tuning: An adaptive tuning of the Q matrix is proposed where an adaptive weighting matrix Qa is computed based on the distance vector between current position and reference at the end of a prediction horizon, helping to minimize tracking errors dynamically.
-
Real-Time Implementation with Backup Model: The closed-loop control scheme can be implemented in real time by using a linear
Backup
Simulation Oriented Model (B-SOM) to simulate the cloth with control signals on each time step, providing updated initial conditions even when real feedback is delayed or unreliable.
Abstract
Robotic cloth manipulation is a relevant challenging problem for autonomous robotic systems. Highly deformable objects as textile items can adopt multiple configurations and shapes during their manipulation. Hence, robots should not only understand the current cloth configuration but also be able to predict the future possible behaviors of the cloth. This paper addresses the problem of indirectly controlling the configuration of certain points of a textile object, by applying actions on other parts of the object through the use of a Model Predictive Control (MPC) strategy, which also allows to foresee the behavior of indirectly controlled points. The designed controller finds the optimal control signals to attain the desired future target configuration. The explored scenario in this paper considers tracking a reference trajectory with the lower corners of a square piece of cloth by grasping its upper corners. To do so, we propose and validate a linear cloth model that allows solving the MPC-related optimization problem in real time. Reinforcement Learning (RL) techniques are used to learn the optimal parameters of the proposed cloth model and also to tune the resulting MPC. After obtaining accurate tracking results in simulation, the full control scheme was implemented and executed in a real robot, obtaining accurate tracking even in adverse conditions. While total observed errors reach the 5 cm mark, for a 30x30 cm cloth, an analysis shows the MPC contributes less than 30% to that value.
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation