ReLU Networks for Model Predictive Control: Network Complexity and Performance Guarantees

arXiv:2601.16764 · eess.SY, cs.LG, cs.SY, math.OC · Submitted 2026-01-23 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "ReLU Networks for Model Predictive Control".

Dev: Here's the gist of the paper. ReLU networks for model predictive control address the open problem of determining network complexity to ensure closed-loop performance guarantees The paper proposes a projection-based method and a non-uniform error framework to establish explicit bounds on ReLU network width and depth for approximating MPC policies with guaranteed closed-loop performance Problem Formulation The work begins by formalizing the problem of determining the network complexity required to guarantee closed-loop performance for a ReLU NN-based MPC policy This involves two main objectives: Problem 1, which seeks an upper bound on network complexity under a uniform error bound, and Problem 2, which explores non-uniform error bounds to enhance convergence to an arbitrarily small invariant set while reducing complexity Uniform Error Bound Analysis For Problem 1, the authors establish explicit bounds on the width and depth of ReLU NNs required for a uniform error bound This involves three main steps: Constraint satisfaction via projection, sharpened closed-loop convergence using a state-dependent Lipschitz continuity property, and deriving complexity bounds based on existing ReLU network approximation theory The final theorem provides explicit bounds for the width and depth in terms of MPC dimensions and constraint set geometry Non-Uniform Error Framework Problem 2 introduces a state-aware scaling mechanism to achieve non-uniform error bounds, which allows approximation errors to scale with the system state This is achieved through state-aware input/output scaling functions that resemble zoom-in/zoom-out quantization The resulting non-uniform error bound function, δ2(x), is defined piecewise based on the state magnitude, which enables exponential convergence to a prescribed invariant set X'inv Complexity Quantification and Trade-offs The analysis demonstrates that the required network complexity strictly improves upon that needed for the uniform error bound when using non-uniform error bounds The complexity bounds are expressed in terms of the GLC of the MPC policy, the desired approximation error δ1, and geometric properties of the constraint set U Numerical examples show that depth is more effective than width for expressive efficiency in approximating the MPC policy The paper concludes by providing explicit guidelines for designing ReLU NN-based MPC policies Key takeaways include prioritizing depth over width for expressive efficiency, noting that a smaller GLC of the MPC policy leads to lower network complexity The state-aware scaling method successfully reduces required network complexity while enhancing closed-loop performance by establishing a trade-off between convergence speed and the size of the terminal invariant set Future work is suggested for extensions to nonlinear, tracking, and time-varying MPC References

1: X.

Rosa: Chen, M.

Dev: Heidarinejad, J.

Rosa: Liu, and P.

Paper summary: Rosa: So, wrapping up this look at "ReLU Networks for Model Predictive Control: Network Complexity and Performance Guarantees," the main thing to take away is that there are now explicit rules for determining network complexity when you use ReLU networks to approximate MPC policies.

Dev: They established that the authors proposed a projection-based method and a non-uniform error framework, which allows them to derive explicit bounds on width and depth with guaranteed closed-loop performance.

Taro: The implication for us is that we can now design these controllers knowing exactly what network size we need before we even start training or deploying anything, which is crucial for real-time systems.

Rosa: It means that by using this state-aware scaling technique, you can get a better approximation rate than you could with just aiming for a uniform error bound across the whole system.

Dev: And they showed that this non-uniform approach strictly improves upon the complexity needed for the uniform error bound, meaning we can actually reduce the required network size while still achieving strong performance guarantees.

Taro: The paper's conclusion points toward prioritizing depth over width for expressive efficiency and suggests that a smaller gain control limit of the MPC policy translates directly into a lower overall network complexity requirement.

Rosa: So, it’s about finding this trade-off between convergence speed and the size of that terminal invariant set when you use these adaptive scaling strategies to get better performance.

Dev: It's a step forward because it moves us beyond just seeing what happens when we use ReLU networks in MPC, toward having a verifiable way to control their size for guaranteed stability.

Conclusion: Rosa: So, this paper, "ReLU Networks for Model Predictive Control: Network Complexity and Performance Guarantees," it's about figuring out how big these neural networks actually need to be to run model predictive control reliably without messing up performance guarantees.

Dev: Right, so they tackle that open problem of determining the network size for these ReLU networks when you want things to stay stable in a closed-loop system.

Taro: The authors propose a way using projection methods and this non-uniform error framework to set explicit limits on the width and depth of those neural networks.

Rosa: What that means is they give us concrete numbers on how wide or deep the network has to be based on how much error we can tolerate.

Dev: They look at two main things: first, finding a top limit for network complexity if we just use a uniform error bound across everything, and second, exploring non-uniform error bounds to get even better results with less complexity.

Taro: The non-uniform part is interesting because they use this state-aware scaling mechanism that makes the approximation error change based on where the system actually is in its state.

Rosa: And it turns out that by using those non-uniform bounds, you can actually get a lower network size than what you'd need just to meet a uniform error goal.

Dev: Exactly, and they show that depth is usually more effective than width when you're trying to get the best approximation for the MPC policy.

Taro: The authors suggest that if your model predictive control policy has a smaller gain control limit, you can also keep your network complexity down.

Rosa: It really boils down to a trade-off between how fast the system converges and how big that terminal invariant set we're aiming for is.

Dev: So, it gives us some clear guidelines on designing these ReLU networks for MPC based on what the paper found about width, depth, and those error bounds.

Taro: But they still leave room to look at tracking systems or time-varying MPC next, which is where things get trickier in the real world.

Tsinghua University · Beijing National Research Center for Information Science & Technology

eess.SY, cs.LG, cs.SY, math.OC

Submitted: 2026-01-23

Updated: 2026-11-11

Code: https://github.com/lixc21/MPC-NN-Complexity

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 91/100

The gist: ReLU networks for model predictive control address the open problem of determining network complexity to ensure closed-loop performance guarantees The paper proposes a projection-based method and a…

Key concepts

Uniform Error Bound Analysis
This method seeks explicit limits on the width and depth of a ReLU network needed to achieve a consistent, small error across all system states. It involves using constraint satisfaction via projection and analyzing state-dependent Lipschitz continuity to derive concrete bounds based on the MPC dimensions.
Non-Uniform Error Framework
This approach introduces state-aware scaling functions that adjust the approximation error based on the current system state. This allows for an error bound that changes with time or position, enabling exponential convergence to a desired invariant set by mimicking zoom-in/zoom-out quantization.
GLC of the MPC Policy
The Generalized Lipschitz Constant (GLC) of the MPC policy measures how sensitive the control law is to changes in system states. A smaller GLC indicates a more stable or predictable policy, which directly translates to a lower required network complexity for approximation.
Depth vs. Width Trade-off
The analysis demonstrates that for approximating MPC policies with ReLU networks, increasing network depth proves more effective for expressive efficiency than increasing width. This suggests that stacking layers provides a better way to capture the necessary nonlinear dynamics within a constrained computational budget.

Terminology

Summary

ReLU networks for model predictive control address the open problem of determining network complexity to ensure closed-loop performance guarantees The paper proposes a projection-based method and a non-uniform error framework to establish explicit bounds on ReLU network width and depth for approximating MPC policies with guaranteed closed-loop performance

Problem Formulation

The work begins by formalizing the problem of determining the network complexity required to guarantee closed-loop performance for a ReLU NN-based MPC policy This involves two main objectives: Problem 1, which seeks an upper bound on network complexity under a uniform error bound, and Problem 2, which explores non-uniform error bounds to enhance convergence to an arbitrarily small invariant set while reducing complexity

Uniform Error Bound Analysis

For Problem 1, the authors establish explicit bounds on the width and depth of ReLU NNs required for a uniform error bound This involves three main steps: Constraint satisfaction via projection, sharpened closed-loop convergence using a state-dependent Lipschitz continuity property, and deriving complexity bounds based on existing ReLU network approximation theory The final theorem provides explicit bounds for the width and depth in terms of MPC dimensions and constraint set geometry

Non-Uniform Error Framework

Problem 2 introduces a state-aware scaling mechanism to achieve non-uniform error bounds, which allows approximation errors to scale with the system state This is achieved through state-aware input/output scaling functions that resemble zoom-in/zoom-out quantization The resulting non-uniform error bound function, δ2(x), is defined piecewise based on the state magnitude, which enables exponential convergence to a prescribed invariant set X'inv

Complexity Quantification and Trade-offs

The analysis demonstrates that the required network complexity strictly improves upon that needed for the uniform error bound when using non-uniform error bounds The complexity bounds are expressed in terms of the GLC of the MPC policy, the desired approximation error δ1, and geometric properties of the constraint set U Numerical examples show that depth is more effective than width for expressive efficiency in approximating the MPC policy

Conclusion and Guidelines

The paper concludes by providing explicit guidelines for designing ReLU NN-based MPC policies Key takeaways include prioritizing depth over width for expressive efficiency, noting that a smaller GLC of the MPC policy leads to lower network complexity The state-aware scaling method successfully reduces required network complexity while enhancing closed-loop performance by establishing a trade-off between convergence speed and the size of the terminal invariant set Future work is suggested for extensions to nonlinear, tracking, and time-varying MPC

References

[1] X. Chen, M. Heidarinejad, J. Liu, and P. D. Christofides, “Distributed economic MPC: Application to a nonlinear chemical process network,” Journal of Process Control, vol. p. 7 22, no. 4, pp. 689–699, 2012

[2] A. N. Venkat, I. A. Hiskens, J. B. Rawlings, and S. J. Wright, “Distributed MPC strategies with application to power system automatic generation control,” IEEE Transactions on Control Systems Technology, vol. p. 0 16, no. 6, pp. 1192–1206, 2008

[3] J. Nubert, J. Kohler, V. Berenz, F. Allg ¨ ower, and S. Trimpe, “Safe and ¨ fast tracking on a robot manipulator: Robust MPC and neural network control,” IEEE Robotics and Automation Letters, vol. p. 0 5, no. 2, pp. 3050–3057, 2020

[4] S. Cheng, L. Li, H.-Q. Guo, Z.-G. Chen, and P. Song, “Longitudinal collision avoidance and lateral stability adaptive control system based on MPC of autonomous vehicles,” IEEE Transactions on Intelligent Transportation Systems, vol. p. 0 21, no. 6, pp. 2376–2385, 2019

[5] K. Nguyen, S. Schoedel, A. Alavilli, B. Plancher, and Z. Manchester, “TinyMPC: Model-predictive control on resource-constrained microcontrollers,” in International Conference on Robotics and Automation (ICRA). p. 0 IEEE, 2024, pp. 1–7

[6] A. Bemporad, M. Morari, V. Dua, and E. N. Pistikopoulos, “The explicit linear quadratic regulator for constrained systems,” Automatica, vol. p. 1 38, no. 1, pp. 3–20, 2002

[7] T. Geyer, F. D. Torrisi, and M. Morari, “Optimal complexity reduction of polyhedral piecewise affine systems,” Automatica, vol. p. 2 44, no. 7, pp. 1728–1740, 2008

[8] P. Tøndel, T. A. Johansen, and A. Bemporad, “Evaluation of piecewise affine control via binary search tree,” Automatica, vol. p. 0 39, no. 5, pp. 945–950, 2003

[9] F. Bayat, T. A. Johansen, and A. A. Jalali, “Using hash tables to manage the time-storage complexity in a point location problem: Application to explicit model predictive control,” Automatica, vol. p. 1 47, no. 3, pp. 571–577, 2011

[10] A. Alessio and A. Bemporad, “A survey on explicit model predictive control,” Nonlinear Model Predictive Control: Towards New Challenging Applications, pp. p. 0 345–369, 2009

[11] A. Bemporad, “Explicit model predictive control,” in Encyclopedia of Systems and Control. p. 0 Springer, 2021, pp. 744–751

[12] A. Tokmak, C. Fiedler, M. N. Zeilinger, S. Trimpe, and J. Kohler, ¨ “Automatic nonlinear mpc approximation with closed-loop guarantees,” IEEE Transactions on Automatic Control, 2025

[13] C. N. Jones and M. Morari, “Polytopic approximation of explicit model predictive controllers,” IEEE Transactions on Automatic Control, vol. p. 0 55, no. 11, pp. 2542–2553, 2010

[14] T. A. Johansen and A. Grancharova, “Approximate explicit constrained linear model predictive control via orthogonal search tree,” IEEE Transactions on Automatic Control, vol. p. 1 48, no. 5, pp. 810–815, 2003

[15] Y. S. Quan and C. C. Chung, “Approximate model predictive control with recurrent neural network for autonomous driving vehicles,” in 58th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE). p. 0 IEEE, 2019, pp. 1076–1081

[16] S. S. P. Kumar, A. Tulsyan, B. Gopaluni, and P. Loewen, “A deep learning architecture for predictive control,” IFAC-PapersOnLine, vol. p. 3 51, no. 18, pp. 512–517, 2018

[17] Y. Lu, D. Li, Z. Xu, and Y. Xi, “Convergence analysis and digital implementation of a discrete-time neural network for model predictive control,” IEEE Transactions on Industrial Electronics, vol. p. 0 61, no. 12, pp. 7035–7045, 2014

[18] D. Wang, Z. J. Shen, X. Yin, S. Tang, X. Liu, C. Zhang, J. Wang, J Rodriguez and M Norambuena, “Model predictive control using artificial neural network for power converters,” IEEE Transactions on Industrial Electronics, vol. p. 0 69, no 4 pp 3689–3699

[19] F. Dong, X. Li, K. You, and S. p. 0 Song, “Standoff tracking using dnnbased mpc with implementation on fpga,” IEEE Transactions on Control Systems Technology, vol. p. 0 31, no 5 pp 1998–2010

[20] S. Lucia, D. Navarro, B. Karg, H. Sarnago, and O. Lucia, “Deep learning-based model predictive control for resonant power converters,” IEEE Transactions on Industrial Informatics, vol. p. 0 17, no 1 pp 409–420

[21] D. Krishnamoorthy, “A sensitivity-based data augmentation framework for model predictive control policy approximation,” IEEE Transactions on Automatic Control, vol. p. 0 67, no. 11, pp. 6090–6097, 2021

[22] S. W. Chen, T. Wang, N. Atanasov, V. Kumar, and M Morari, “Large scale model predictive control with neural networks and primal active sets,” Automatica, vol. p. 0 135 p 109947, 2022

[23] S. Chen, K. Saulnier, N. Atanasov, D. D. Lee, V. Kumar, G J Pappas and M Morari, “Approximating explicit model predictive control using constrained neural networks,” in Annual American control conference (ACC). p. 0 IEEE, 2018 pp 1520–1527

[24] M. Hertneck, J. Kohler, S. Trimpe, and F. Allg ¨ ower, “Learning an ¨ approximate model predictive controller with guarantees,” IEEE Control Systems Letters, vol. p. 0 2, no 3 pp 543–548

[25] F. Fabiani and P. J. Goulart, “Reliably-stabilizing piecewise-affine neural network controllers,” IEEE Transactions on Automatic Control, vol. p. 2 68, no 9 pp 5201–5215, 2022

[26] R. Schwan, C. N. Jones, and D Kuhn, “Stability verification of neural network controllers using mixed-integer programming,” IEEE Transactions on Automatic Control, vol. p. 0 68, no 12 pp 7514–7529

[27] C. Gonzalez, H. Asadi, L. Kooijman, and C. P. Lim, “Neural networks for fast optimisation in model predictive control: A review,” arXiv preprint arXiv:2309. p. 002668, 2023

[28] M. S. B Karg, “Efficient approximations of model predictive control laws via deep learning,” Ph. p. 0D. dissertation, Technische Universitat¨ Dortmund, 2023

[29] Z. Shen, H. Yang, and S Zhang, “Optimal approximation rate of ReLU networks in terms of width and depth,” Journal de Mathematiques Pures ´ et Appliquees ´, vol. p. 0 157, pp. 101–135, 2022

[30] J. A. Opschoor, C. Schwab, and J Zech, “Exponential relu dnn expression of holomorphic maps in high dimension,” Constructive Approximation, vol. p. 0 55, no 1 pp 537–582, 2022

[31] M. Ali and A. Nouy, “Approximation of smoothness classes by deep rectifier networks,” SIAM Journal on Numerical Analysis, vol. p. 5 59, no. 6 pp 3032–3051, 2021

[32] J. Berner, P. Grohs, G Kutyniok, and P Petersen, “The modern mathematics of deep learning,” arXiv preprint arXiv:2105. p. 004026, pp. 86–114, 2021

[33] R. DeVore, B Hanin, and G Petrova, “Neural network approximation,” Acta Numerica, vol. p. 0 30 pp 327–444, 2021

[34] F. Borrelli, A. Bemporad, and M. Morari, Predictive control for linear and hybrid systems. p. 1 Cambridge University Press, 2017

[35] J. B. Rawlings, D Q Mayne, and M Diehl, Model predictive control: theory computation design. p. 0 Nob Hill Publishing Madison WI, 2017 vol 2

[36] X. Li and K. You, “ReLU neural networks for approximating model predictive control: Complexity and stability guarantees,” in American Control Conference (ACC). p. 0 IEEE, 2025 pp 27–32

[37] K. You, W. Su, M. Fu, and L Xie, “Attainability of the minimum data rate for stabilization of linear systems via logarithmic quantization,” Automatica, vol. p. 1 47, no 1 pp 170–176, 2011

[38] Z. Shen, H. Yang, and S Zhang, “Deep network approximation characterized by number of neurons,” arXiv preprint arXiv:1906. p. 005497, 2019

[39] Y. Wang and S Boyd, “Fast model predictive control using online optimization,” IEEE Transactions on Control Systems Technology, vol. p. 0 18, no 2 pp 267–278, 2009

[40] V. Kucera, “The discrete riccati equation of optimal control,” ˇ Kybernetika, vol. p. 7 8, no. 5 pp 430–447, 1972

[41] A. Paszke, S. Gross, F. Massa, A. Lerer, J Bradbury, G Chanan et al., “PyTorch: An imperative style, high-performance deep learning library,” Advances in Neural Information Processing Systems, vol 32, 2019

[42] Gurobi Optimization LLC, “Gurobi Optimizer Reference Manual,” 2024 [Online]. Available: https://www. p. 7gurobi.com

[43] B. Hanin, “Universal function approximation by deep neural nets with bounded width and ReLU activations,” Mathematics, vol. p. 5 7, no 10 p 992, 2019

[48] Lemma 12 ([29, Theorem 1.1]). Given f ∈ C([0, 1]nx), for any L′', N′ ∈ N+, and for p ∈ [1,∞), there exists a ReLU NN ϕ with width C1 max(nx⌊N′/nx⌋, N' + 2) and depth 11L' + C2, such that f−ϕLp([0,1]nx) ≤ 13.

[49] Parameter count of the NNs shows that the number of parameters increases quadratically with the width and linearly with the depth

[50] The best choice in practice is not to arbitrarily increase the depth, but to consider the total computational cost and choose an appropriate depth and width

[51] Fig. 8 shows that as the depth and width of the NN increase, the MSE decreases

[52] The parameter count in Table I shows that the number of parameters increases quadratically with the width and linearly with the depth

[53] By comparing different depths at the same width, a shallow network (e.g., nd = 1) has a considerably higher MSE than deeper ones

[54] The parameter count in Table I shows that the number of parameters increases quadratically with the width and linearly with the depth

[55] We also conducted experiments on the state-aware scaling method, using the same model (hidden width 16 and hidden depth 2) and parameter settings as the absolute error method, with ¯δ = 1, c8 = 0.5, and varying δ

[56] As shown in Fig. 9, as δ decreases, both the test loss and the average cost increase, but the average x over the last 12 steps decreases

Table I

The parameter count of the NNS shows that the number of parameters increases quadratically with the width and linearly with the depth

nw = 8 nw = 12 nw = 18 nw = 24 nw = 32 nd = 1 33 49 nd = 2 105 nd = 3 177 nd=4 249 nd=5 321 nd=6 393

nw = nw = nw = nw = nw

nd | | | | |

Fig. 8

MSE (scaled by 10 4) of the NNs

Fig. 9

Closed-loop Performance under Different δ loss and the average cost increase, but the average x over the last 12 steps decreases

Fig. 7

Oscillating Masses System

Fig. 4

The non-uniform scaling approach (The definitions of uempc(·) and uenn(·) are given in (24) and (30)

Fig. 5

An example of β(x) = δ2(x)/¯δ and the input scaling function T(x).

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. State-aware scaling for non-uniform error bounds can be used to design a state-aware scaling function T(x) that stretches or compresses input space based on the current state norm, allowing static ReLU networks to achieve non-uniform approximation errors that decay to zero as x approaches the origin.

  2. The improved AI system can guarantee closed-loop convergence to an arbitrarily small invariant C-set X prime by using a non-uniform error bound delta2(x) that is state dependent, which is achieved through scaling both the input and output of the ReLU network.

  3. Explicit bounds on network width and depth can be derived for ReLU NN based MPC policies with certifiable closed-loop performance, providing concrete design guidelines by showing that depth over width is more effective for expressive efficiency in these systems.

Sources

Related papers