Stable Transformer-Actor-Critic Model Predictive Control: A Contraction Analysis Approach
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: We're looking at the paper: "Stable Transformer-Actor-Critic Model Predictive Control".
Dev: Here's the gist of the paper. Actor-Critic Model Predictive Control (MPC) effectively addresses complex, non-convex control problems, but guaranteeing the closed-loop stability of sequence-based learning models within these pipelines remains challenging.
Rosa: This paper introduces a novel Transformer-Actor-Critic MPC architecture with formal robustness guarantees.
Dev: Theoretical Foundations of Stability The authors prove that Transformer networks can satisfy global incremental Input-to-State Stability (δISS) with respect to the induced infinity norm.
Rosa: They leverage Riemannian contraction theory to analyze the interconnected dynamics between the physical plant and the predictive neural network.
Title and authors: Rosa: So to recap, this paper is about taking that known difficulty in making sequence-based learning models stable within Model Predictive Control and proposing a new architecture with formal robustness guarantees. It’s titled "Stable Transformer-Actor-Critic Model Predictive Control: A Contraction Analysis Approach."
Dev: The focus is on using contraction analysis to prove stability, specifically incremental Input-to-State Stability, or deltaISS, for the Transformer networks involved. They use Riemannian contraction theory to link the physical system dynamics with the neural network dynamics.
Taro: I’m thinking about what this means for us in terms of deployment; if it’s stable under these theoretical conditions, does that mean we can trust it more when we deploy it on hardware where things aren't perfectly modeled?
Rosa: They are showing that by integrating these theoretical bounds as a training regularizer, you get a policy that is certifiably robust. They are doing this validation on a nonlinear three dee drone model executing target-reaching and obstacle avoidance maneuvers, which shows it works in complex scenarios <ref:2606.20197#pg1,on a nonlinear 3D drone model executing target-reaching and obstacle avoidance>.
Dev: The paper shows they can achieve deltaISS with respect to the induced infinity norm, which is a very strict mathematical condition for stability analysis. It means the state deviation stays bounded by some function of how far you start and how much input error you have during training.
Taro: Bounded by a function of the initial state deviation and input error—that’s helpful because it sets a hard limit on how much surprise the system can handle before it diverges. Does that bound apply to the kind of disturbances we usually deal with?
Rosa: They show that when you enforce this theoretical bound during training, you get a control policy that is both highly performant and certifiably robust. The validation shows the stable network robustly converges to the target location even in obstacle-cluttered environments under severe disturbances.
Dev: So, it’s not just about learning; it’s about constraining the learning process itself so that you get a safe control policy. This moves beyond just optimizing performance to ensuring safety during the learning phase.
The paper's summary: Rosa: Let's look closer at what they actually propose in "Stable Transformer-Actor-Critic Model Predictive Control: A Contraction Analysis Approach." They start by defining a discrete-time nonlinear system, which is the standard setup for MPC, including the state x sub k plus one equals Phi of x sub k and u sub k plus d sub k.
Dev: They then introduce a specific definition called deltaISS with respect to the induced infinite norm. This definition states that if you have two state trajectories, one starting at x sub one zero and another at x sub two zero, the deviation between them over time, delta x sub k equals x sub one k minus x sub two k, is bounded by some function of the initial deviation and input deviation <ref:2606.20197#pg2>.
Taro: So it’s mathematically bounding the error between two possible outcomes of the system when they start slightly differently or receive slightly different inputs. That sounds like a very rigorous way to define safety for a learning loop.
Rosa: Exactly. Then they move into analyzing the Transformer layer itself by interpreting it as a discrete dynamical system, which follows this form: z sub k plus one equals alpha two g of alpha one z sub k plus A of z sub k and p sub k plus f of g of alpha one z sub k plus A of z sub k.
Dev: They define what those components mean—alpha one and alpha two are contraction rates less than one, f is a network with one-Lipschitz continuous activation functions like tanh or ReLU, and A is the multi-head attention mechanism <ref:2606.20197#pg1>. This lets them analyze the layer mathematically.
Taro: So they’re breaking down the Transformer block by block to see how it contributes to that overall stability condition, which gives us insight into where potential instability might creep in within the network structure itself.
Rosa: And then, they integrate these theoretical bounds as a training regularizer. This is where the architecture gets its name; they are using this mathematical proof to constrain the learning process so that you get a certifiably robust policy output from the Actor-Critic MPC setup.
The paper's improvements: Dev: The paper suggests several specific ways to improve this framework, moving beyond just proving stability. One big thing is how they handle the constraints in the cost function. They relax hard boundaries and instead integrate them into the objective function using a smooth Softplus penalty.
Taro: That makes sense from a real-world standpoint because if you have hard boundaries, any tiny error in calculation can cause an abrupt jump in the control signal, which is dangerous. A smooth penalty should prevent that kind of jittery behavior during operation.
Rosa: Right. They also use attention mechanism integration for MPC parameter shaping. This allows the Transformer networks to predict context-aware MPC weights directly from state trajectories, which dynamically shapes the local optimization landscape by adjusting linear and quadratic coefficients of the cost function.
Dev: So the attention mechanism isn't just there for capturing temporal dependencies; it’s actively shaping *how* the optimization problem is solved at every single time step based on where we are in the trajectory. That’s a powerful way to make the optimization landscape more manageable for a neural network.
Taro: If it can dynamically adjust those weights based on context, that means when you're in a rough spot, the network can adapt its planning strategy immediately rather than relying on pre-programmed coefficients that might be too rigid. That sounds like good recovery behavior.
Rosa: They also show how to evaluate this system using mixed sensitivity norms that bound the Euclidean solver sensitivity into the physical metric, which gives us a way to measure how sensitive the solver is to errors in terms of infinity-to-two.
Conclusion: Dev: So wrapping up, the big picture is that this framework yields a control policy that is both highly performant and certifiably robust because they enforced closed-loop contraction stability during training. They achieved guaranteed incremental Input-to-State Stability with respect to the induced infinity norm.
Rosa: It means we can trust these systems more in dangerous situations because the math backs the performance, especially when dealing with disturbances that are severe. The drone model validation showed it robustly converges even in obstacle-cluttered environments under maximal disturbance conditions.
Taro: From my side, what’s interesting is that the paper shows they can achieve this stability while still having rotational agility and not just being a perfectly rigid, predictable path follower. They mention exploring state-dependent contraction metrics to mitigate trade-offs between strict global stability bounds and rotational agility in future work.
Dev: That makes sense. The authors themselves noted that one limitation is the trade-off between achieving strict global stability bounds and maintaining rotational agility, which they plan to address by looking at state-dependent contraction metrics. They also mentioned plans to extend this framework to output-feedback settings using partial contraction theory in the future work.
Rosa: So, in short, this paper provides a control policy that is both highly performant and certifiably robust through a rigorous mathematical analysis of stability. It’s a solid piece of work for anyone looking to move learning-based control into more safety-critical applications.
Dev: It certainly moves the needle by providing that formal guarantee, which is what we need when we're moving these systems out of the simulation and onto hardware where failure isn't an option. We’ll keep watching this space for how they tackle those state-dependent metrics next.
Antonio Marino, Valerio Modugno, Marco Cognetti
University of Cambridge · University College London · LAAS-CNRS, Universite de Toulouse, CNRS, UPS, Toulouse, France
cs.RO
Submitted: 2026-06-18
Updated: 2026-11-15
Project page: https://seaingant.github.io/stac-mpc
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 99/100
The gist: Actor-Critic Model Predictive Control (MPC) effectively addresses complex, non-convex control problems…
Key concepts
- Input-to-State Stability (ISS)
- ISS is a mathematical property ensuring that if you introduce an external disturbance (input) into the system, the resulting state of the system will remain bounded. The paper proves that their Transformer network satisfies this property, meaning small changes in input lead only to small, predictable changes in the output state.
- Riemannian Contraction Theory
- This is a mathematical tool used to analyze how dynamics behave on curved spaces. In this context, it helps the authors connect the dynamics of the physical drone plant with their neural network. It allows them to mathematically prove that errors between the system and its prediction shrink over time, leading to stability.
- Closed-Loop Contraction Stability
- This condition is a requirement for guaranteed performance. The authors enforce this during training by ensuring the system dynamics contract when operating in a closed loop. This contraction dictates that the error state vector shrinks, which is the core mechanism used to guarantee robust control.
Terminology
Summary
Actor-Critic Model Predictive Control (MPC) effectively addresses complex, non-convex control problems, but guaranteeing the closed-loop stability of sequence-based learning models within these pipelines remains challenging. p. 0 This paper introduces a novel Transformer-Actor-Critic MPC architecture with formal robustness guarantees. p. 0
Theoretical Foundations of Stability
The authors prove that Transformer networks can satisfy global incremental Input-to-State Stability (δISS) with respect to the induced infinity norm. They leverage Riemannian contraction theory to analyze the interconnected dynamics between the physical plant and the predictive neural network. p. 0 The framework integrates these theoretical bounds as a training regularizer to yield a certifiably robust policy. p. 0
Architecture and Training Signal Construction
The proposed hybrid architecture uses an initial MLP to embed history into a latent sequence zk. A core Transformer block subsequently filters this sequence to capture temporal dependencies, yielding a refined representation pk that a terminal MLP projects and reshapes into the cost parameterization Y. p. 1 To ensure smooth gradients and bypass piecewise active set discontinuities, inequality constraints are relaxed from hard boundaries and integrated into the objective function via a smooth Softplus penalty. p. 1
Where the Gains Come From
The stability analysis relies on deriving a coupled small-gain condition for the physical plant by enforcing closed-loop contraction stability during training. The system is contractive if and only if the spectral radius of the joint mixed-norm error state vector Ek is less than 1. p. 4 This condition dictates that (1 − ρmpc)(1 − Aδ) − γzγx > 0. p. 4
Validation and Results
Numerical validation on a highly nonlinear 3D drone model demonstrated that enforcing this theoretical bound during training yields a robust control policy. The stable network robustly converges to the target location in both the unconstrained and obstacle-cluttered environments under severe disturbances. p. 4 Conversely, the unregularized vanilla network systematically fails to reject the noise leading to severe positional divergence. p. 4 The stable network achieved a 0% collision rate in the obstacle avoidance scenario under maximal disturbance conditions. p. 4
Conclusion and Future Work
The paper concludes that the proposed framework yields a control policy that is both highly performant and certifiably robust. Future work will explore state-dependent contraction metrics to mitigate trade-offs between strict global stability bounds and rotational agility. p. 5 They also plan to extend this framework to output-feedback settings leveraging partial contraction theory. p. 5
REFERENCES
[1] D. P. Bertsekas, “Model predictive control and reinforcement learning: A unified framework based on dynamic programming,” IFACPapersOnLine, vol. p. 5 58, no. 18, pp. 363–383, 2024
[2] L. Hewing, K. P. Wabersich, M. p. 5 Menner, and M. p. 5 N. Zeilinger, “Learning-based model predictive control: Toward safe learning in control,” Annual Review of Control, Robotics, and Autonomous Systems, vol. p. 5 3, pp. 269–296, 2020
[3] K. Dzhumageldyev, F. p. 5 Airaldi, and A. p. 5 Dabiri, “Safe model-based reinforcement learning via model predictive control and control barrier functions,” arXiv preprint arXiv:2512. p. 504856, 2025
[4] F. Bonassi, A. p. 5 La Bella, M. p. 5 Farina, and R. p. 5 Scattolini, “Nonlinear mpc design for incrementally iss systems with application to gru networks,” Automatica, vol. p. 5 159, p. 111381, 2024
[5] M. Alamir, “Contraction-based nonlinear model predictive control formulation without stability-related terminal constraints,” Automatica, vol. p. 5 75, pp. 288–292, 2017
[6] L. Wei, R. McCloy, J. Bao, and J. p. 5 Cranney, “Discrete-time contraction constrained nonlinear model predictive control using graph-based geodesic computation,” AIChE Journal, vol. p. 5 68, no. 11, p. e17830, 2022
[7] A. Romero, Y. Song, and D. p. 5 Scaramuzza, “Actor-critic model predictive control,” in 2024 IEEE International Conference on Robotics and Automation (ICRA). p. 5 IEEE, 2024, pp. p. 5 14 777–14 784
[8] E. Adabag, M. Greiff, J. Subosits, and T. p. 5 Lew, “Differentiable model predictive control on the gpu,” arXiv preprint arXiv:2510. p. 506179, 2025
[9] B. Amos, I. Jimenez, J. p. 5 Sacks, B. Boots, and J. p. 5 Z. Kolter, “Differentiable mpc for end-to-end planning and control,” Advances in neural information processing systems, vol. p. 5 31, 2018
[10] L. Amatucci, J. p. 5 Sousa-Pinto, G. p. 5 Turrisi, D. p. 5 Orban, V. Barasuol, and C. p. 5 Semini, “Primal-dual ilqr for gpu-accelerated learning and control in legged robots,” IEEE Robotics and Automation Letters, vol. p. 5 11, no. 1, pp. 1010–1017, 2025
[14] Y. D. Zhong, T. Zhang, A. Chakraborty, and B. p. 5 Dey, “A neural ode interpretation of transformer layers,” CoRR, vol. p. 5 abs/2212.06011, 2022
[15] A. R. Abella, J. P. Silvestre, and P. p. 5 Tabuada, “The asymptotic behavior ´ of attention in transformers,” arXiv preprint arXiv:2412. p. 502682, 2024
[16] W. D’Amico, A. p. 5 La Bella, and M. p. 5 Farina, “An incremental input-tostate stability condition for a class of recurrent neural networks,” IEEE Transactions on Automatic Control, vol. p. 5 69, no. 4, pp. 2221–2236, 2023
[17] A. Marino, C. Pacchierotti, and P. p. 5 R. Giordano, “Input state stability of gated graph neural networks,” IEEE Transactions on Control of Network Systems, vol. p. 5 11, no. 4, pp. 2052–2063, 2024
[18] H. Tsukamoto, S. p. 5-J. Chung, and J. p. 5-J. E. Slotine, “Contraction theory for nonlinear stability analysis and learning-based control: A tutorial overview,” Annual Reviews in Control, vol. p. 5 52, pp. 135–169, 2021
[19] F. Bayer, M. Burger, and F. p. 5 Allg ¨ ower, “Discrete-time incremental iss: ¨A framework for robust nmpc,” in 2013 European Control Conference (ECC). p. 5 IEEE, 2013, pp. p. 5 2068–2073
[20] E. Parisotto, F. p. 5 Song, J. Rae, R. Pascanu, C. p. 5 Gulcehre, S. p. 5 Jayakumar, M. p. 5 Jaderberg, R. p. 5 L. Kaufman et al. p. 5, “Stabilizing transformers for reinforcement learning,” in International conference on machine learning PMLR, 2020
[21] J. Xu, X. Sun, Z. Zhang, G. Zhao, and J. p. 5 Lin, “Understanding and improving layer normalization,” Advances in neural information processing systems, vol. p. 5 32, 2019
[22] G. Russo, M. Di Bernardo, and E. p. 5 D. Sontag, “A contraction approach to the hierarchical analysis and design of networked systems,” IEEE Transactions on Automatic Control, vol. p. 5 58, no. 5, pp.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Attention mechanism integration for MPC parameter shaping allows Transformer networks to predict context-aware MPC weights from state trajectories, which shapes the local optimization landscape by dynamically adjusting linear and quadratic coefficients of the cost function.
-
The framework yields a control policy that is both highly performant and certifiably robust by enforcing closed-loop contraction stability during training, as demonstrated by
By enforcing this closedloop contraction stability during training, our framework yields a control policy that is both highly performant and certifiably robust.
-
The system can achieve guaranteed incremental Input-to-State Stability (deltaISS) with respect to the induced infinity norm, allowing the architecture to be mathematically constrained to satisfy global deltaISS.
-
The closed-loop MPC policy is globally exponentially contractive on a time-varying M-weighted Riemannian manifold, with a strict, uniform contraction rate bounded by rho mpc less than one.
-
The system can be evaluated using mixed sensitivity norms that bound the Euclidean solver sensitivity into the physical metric, as shown by
SY ≤ p lambda max(M) 2 mu2 B2 J infinity-to-2.
-
The drone model can robustly converge to the target location in both unconstrained and obstacle-cluttered environments, maintaining flight stability despite severe uniform observation noise and process disturbances where unregularized baselines catastrophically fail.
Sources
- Safe model-based Reinforcement Learning via Model Predictive Control and Control Barrier Functions
- Differentiable Model Predictive Control on the GPU
- Transformer Is Inherently a Causal Learner
- The Asymptotic Behavior of Attention in Transformers
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation