Online Neural Network Training via Closed-Loop Model-Free Tracking Control: An Empirical Study

arXiv:1905.02230 · eess.SY, cs.SY, math.OC · Submitted 2019-05-06 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: I'm Rosa.

Dev: And I'm Dev, here with Taro.

Rosa: We're looking at the paper: "Online Neural Network Training via Closed-Loop Model-Free Tracking Control".

Dev: Here's the gist of the paper. Online learning of neural networks based on a model-free control algorithm explores using a model-free-based control law to train artificial neural networks in a supervised learning context by tuning synaptic weights as a feedback control tracking problem Training a neural network consists of tuning its internal weights in order to learn a mapping function from inputs to outputs and eventually examine what the model predicts Besides classical tuning techniques some connections between adaptive control and optimization methods have been pointed out recently that highlight a certain equivalence between using tools from the adaptive control field and solving problems in the machine learning field The motivation of this work is to propose a strategy to tune neural networks using the so-called model-free control algorithm in the context of supervised learning Principle of Para-Model Control The para-model control is an application Cπ: (y, y∗) 7→ u whose purpose is to control the output y of (1) following an output reference y∗ For any discrete moment tk, k ∈ IN∗ one defines the discrete controller Cπ: (y, y∗) 7→ u as an integrator associated to a numerical series (Ψk)k∈IN such as symbollicly uk = Cakπ (yk, y∗k) = Ψk Application to Neural Network Training The proposed strategy updates each synaptic weight such as W1 = Ca1π (y, ytrain), W2 = Ca2π (y, ytrain), and so on This allows configuring the neural network in such manner that asymptotically the output y remains as close as possible to ytrain A filter is associated to each Wi in order to include a dynamic regarding the proper use of the Cπ controllers Empirical Results and Performance Numerical results illustrate the dynamical learning process and an example of classifier that show very promising properties of our proposed approach A great tracking of the output y has been observed despite the different changes of the training data as well as modifications of the topology of the network The set of data is properly classified according to the boolean comparison with y Remark that since the proposed control-based training algorithm deals with dynamical systems and sweeps the training data through low pass filtering, the partition of the classifier via y corresponds indeed to the ’filtered’ averaged value of the output training data This paper presented an application of the model-free-based control methodology in the field of artificial neural networks Further works will include the formalization of our proposed approach as well as investigations regarding the application of our proposed algorithm to large scale neural networks The paper is organized as follows, reviewing the para-model approach in Section 2, presenting an example of distributed model-free-based control in Section 3, and presenting numerical results in Section 5 The implementation of the control scheme is depicted in Fig.

Rosa: 1 The evolution of the weights Wi and the controlled output y versus iterations when the network is subjected to changes xtrain1 = 0.15, xtrain2 = 0.8 at k = k1 and then ytrain = 0.6 at k = k3 as well as also subjected to a modification of the neural network topology (setting W7 = 0) at k = k2 The set of Cπ-parameters of the controller is defined as the set of the tuning coefficients Kp, Ki, kα, and kβ The control law associated to each variable xj is given by xj = Caⱼπ (yj, bj) The set of references includes G.

Dev: Montavon, W.

Rosa: Samek, and K.-R.

Paper summary: Rosa: So to wrap up on "Online Neural Network Training via Closed-Loop Model-Free Tracking Control: An Empirical Study," this work shows a way to use model-free control to tune neural network weights online in a supervised learning context <ref:1905.02230#pg1>. It suggests that treating weight tuning as a tracking problem with para-model control can lead to effective dynamical learning <ref:1905.02230#pg1>.

Dev: The authors, including L. Michel, explore this at the intersection of control and optimization in machine learning <ref:1905.02230#pg2>. The paper demonstrates that even when things change—data shifts or network topology modifications—the system can track the training data output effectively <ref:1905.02230#pg1>.

Taro: For someone just listening, what does this mean practically? It means we're not just training a static network; we are creating a control loop that constantly adjusts the network parameters based on what it's currently seeing <ref:1905.02230#pg1>. That constant adjustment is key to handling dynamic environments.

Rosa: And the key distinction they make is that this approach involves sweeping the training data through low pass filtering, so the final classifier partition you get reflects an averaged value of the training data output <ref:1905.02230#pg1>. That’s a specific way they handle noise or rapid changes during learning.

Dev: The implications are that we can use these model-free control techniques for online, adaptive AI systems where the underlying model parameters need to evolve constantly based on real-time input <ref:1905.02230#pg1>. But we have to keep in mind what they state: this specific approach is dealing with dynamical systems and using low pass filtering, which means that the partition of the classifier via y corresponds indeed to the 'filtered' averaged value of the output training data <ref:1905.02230#pg1>.

Taro: So, while it shows strong tracking properties under dynamic conditions like topology changes and data variations <ref:1905.02230#pg1>, we still have to look at how this method scales for very large neural networks in the real world <ref:1905.02230#pg1>. That's where the next investigation needs to happen.

Conclusion: Rosa: So, this paper is about using model free control to train neural networks online in a supervised learning setting by tuning synaptic weights as if it were a feedback control tracking problem <ref:1905.02230#pg1>.

Dev: It essentially treats weight tuning like you're trying to follow a target, where the controller is model free, and the goal is to track the training data output while updating those synaptic weights <ref:1905.02230#pg1>.

Taro: The core idea seems to be this para model control which defines a controller that takes the current output and a desired reference and makes a move for each discrete moment <ref:1905.02230#pg1>.

Rosa: And they apply this by updating each synaptic weight sequentially, like weight one is controlled by its own para model based on the current output and the training data <ref:1905.02230#pg1>.

Dev: That means asymptotically, the network output stays as close as possible to whatever you're training it on <ref:1905.02230#pg1>. But there's a filter associated with each weight, which is important for how fast and how smoothly things happen <ref:1905.02230#pg1>.

Taro: The results show that this method handles changes really well, even when the training data itself shifts or the network structure gets modified during learning <ref:1905.02230#pg1>.

Rosa: They even say that because they're sweeping the training data through some low pass filtering, the final classification partition you get is essentially a filtered average of what it saw during training <ref:1905.02230#pg1>.

Dev: That’s a big deal for robustness, but I have to ask about the practicalities—how stable is this loop when things go really wrong in real time?

Taro: That's exactly what we need to know. Can it handle those sudden changes gracefully when we push it outside of the lab environment?

Rosa: We’ll see. This work is setting up a framework for how AI can adapt its own internal structure based on continuous feedback <ref:1905.02230#pg1>. What happens next is figuring out if this control strategy can be scaled up to massive neural networks where the number of weights gets huge?

Loé MICHEL

Ecole centrale de Nantes-LS2N · UMR 6004 CNRS

eess.SY, cs.SY, math.OC

Submitted: 2019-05-06

Updated: 2026-11-16

Comments: 13 pages, 6 figures, Submitted to the 9th Annual Learning for Dynamics & Control Conference

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

Importance score: 65/100

The gist: Online learning of neural networks based on a model-free control algorithm explores using a model-free-based control law to train artificial neural networks in a supervised learning context by tuning…

Key concepts

Model-free Control Algorithm
This is a control strategy that trains the neural network without needing an explicit mathematical model of how the network works internally. Instead, it uses feedback from the actual output to adjust the synaptic weights iteratively, similar to how a physical system learns through trial and error.
Para-Model Control ($ ext{C}_ au$)
This is a specific type of controller designed to follow a desired output reference ($y^*$). It works by defining an integrator associated with a numerical series ($ orall k$) that calculates the necessary control action based on the current output and the target value.
Synaptic Weight Tuning
In this context, tuning synaptic weights is equivalent to adjusting the parameters of a physical system. The controller updates each weight ($W_i$) based on its error relative to the training data, aiming to minimize this error over time so the network's output matches the desired target.
Feedback Control Tracking Problem
This frames neural network training as a control task where the goal is to keep the actual network output ($y$) as close as possible to a desired target output ($y_{train}$). The control law dictates how weights are adjusted in real-time based on this tracking error.

Terminology

Summary

Online learning of neural networks based on a model-free control algorithm explores using a model-free-based control law to train artificial neural networks in a supervised learning context by tuning synaptic weights as a feedback control tracking problem

Introduction to the Approach

Training a neural network consists of tuning its internal weights in order to learn a mapping function from inputs to outputs and eventually examine what the model predicts Besides classical tuning techniques some connections between adaptive control and optimization methods have been pointed out recently that highlight a certain equivalence between using tools from the adaptive control field and solving problems in the machine learning field The motivation of this work is to propose a strategy to tune neural networks using the so-called model-free control algorithm in the context of supervised learning

Principle of Para-Model Control

The para-model control is an application Cπ: (y, y∗) 7→ u whose purpose is to control the output y of (1) following an output reference y∗ For any discrete moment tk, k ∈ IN∗ one defines the discrete controller Cπ: (y, y∗) 7→ u as an integrator associated to a numerical series (Ψk)k∈IN such as symbollicly uk = Cakπ (yk, y∗k) = Ψk

Application to Neural Network Training

The proposed strategy updates each synaptic weight such as W1 = Ca1π (y, ytrain), W2 = Ca2π (y, ytrain), and so on This allows configuring the neural network in such manner that asymptotically the output y remains as close as possible to ytrain A filter is associated to each Wi in order to include a dynamic regarding the proper use of the Cπ controllers

Empirical Results and Performance

Numerical results illustrate the dynamical learning process and an example of classifier that show very promising properties of our proposed approach A great tracking of the output y has been observed despite the different changes of the training data as well as modifications of the topology of the network The set of data is properly classified according to the boolean comparison with y Remark that since the proposed control-based training algorithm deals with dynamical systems and sweeps the training data through low pass filtering, the partition of the classifier via y corresponds indeed to the ’filtered’ averaged value of the output training data

Conclusion and Future Work

This paper presented an application of the model-free-based control methodology in the field of artificial neural networks Further works will include the formalization of our proposed approach as well as investigations regarding the application of our proposed algorithm to large scale neural networks The paper is organized as follows, reviewing the para-model approach in Section 2, presenting an example of distributed model-free-based control in Section 3, and presenting numerical results in Section 5 The implementation of the control scheme is depicted in Fig. 1 The evolution of the weights Wi and the controlled output y versus iterations when the network is subjected to changes xtrain1 = 0. p. 6

Where the paper says it

Online learning of neural networks based on a model-free control algorithm 7 (a) Weights Wi (b) Output y and output reference y ∗ Fig. Evolution of the weights Wi and the controlled output y versus iterations when the network is subjected to the changes x train 1 = 0.

p. 6
Where the paper says it

Online learning of neural networks based on a model-free control algorithm 7 (a) Weights Wi (b) Output y and output reference y ∗ Fig. Evolution of the weights Wi and the controlled output y versus iterations when the network is subjected to the changes x train 1 = 0.

15, xtrain2 = 0.8 at k = k1 and then ytrain = 0. p. 4
Where the paper says it

Online learning of neural networks based on a model-free control algorithm 5 W1 = C {Kp 1,Ki 1,kα 1,kβ 1} π (y, ytrain), W2 = C {Kp 2,Ki 2,kα 2,kβ 2} π (y, ytrain), . Wq = C {Kp q,Ki q,kα q,kβq} π (y, ytrain).

p. 4
Where the paper says it

Online learning of neural networks based on a model-free control algorithm 5 W1 = C {Kp 1,Ki 1,kα 1,kβ 1} π (y, ytrain), W2 = C {Kp 2,Ki 2,kα 2,kβ 2} π (y, ytrain), . Wq = C {Kp q,Ki q,kα q,kβq} π (y, ytrain).

6 at k = k3 as well as also subjected to a modification of the neural network topology (setting W7 = 0) at k = k2 The set of Cπ-parameters of the controller is defined as the set of the tuning coefficients Kp, Ki, kα, and kβ The control law associated to each variable xj is given by xj = Caⱼπ (yj, bj) The set of references includes G. p. 2
Where the paper says it

Proposed para-model scheme to control a nonlinear system. Define the set of the Cπ-parameters of the controller as the set of the tuning coefficients {Kp, Ki , kα, kβ} 2 .

p. 2
Where the paper says it

Proposed para-model scheme to control a nonlinear system. Define the set of the Cπ-parameters of the controller as the set of the tuning coefficients {Kp, Ki , kα, kβ} 2 .

Montavon, W. p. 7
Where the paper says it

References 1. Montavon, W.

p. 7
Where the paper says it

References 1. Montavon, W.

Samek, and K. p. 7
Where the paper says it

Montavon, W. Samek, and K.

p. 7
Where the paper says it

Montavon, W. Samek, and K.

-R. M¨uller in Digital Signal Processing C. p. 7
Where the paper says it

Methods for interpreting and understanding deep neural networks. Digital Signal Processing, 73:1–15, 2018.

p. 7
Where the paper says it

Methods for interpreting and understanding deep neural networks. Digital Signal Processing, 73:1–15, 2018.

C. Aggarwal in Neural Networks and Deep Learning M. p. 7
Where the paper says it

Digital Signal Processing, 73:1–15, 2018. Neural Networks and Deep Learning.

p. 7
Where the paper says it

Digital Signal Processing, 73:1–15, 2018. Neural Networks and Deep Learning.

Fliess and C. p. 7
Where the paper says it

In 2019 IEEE 58th Conference on Decision and Control (CDC), pages 3724–3740, 2019. Fliess and C.

p. 7
Where the paper says it

In 2019 IEEE 58th Conference on Decision and Control (CDC), pages 3724–3740, 2019. Fliess and C.

Join in Model-free control L. p. 0
Where the paper says it

Online learning of neural networks based on a model-free control algorithm Lo¨ıc MICHEL Ecole centrale de Nantes-LS2N, UMR 6004 CNRS, Nantes, France ´ loic. michel@ec-nantes.

p. 0
Where the paper says it

Online learning of neural networks based on a model-free control algorithm Lo¨ıc MICHEL Ecole centrale de Nantes-LS2N, UMR 6004 CNRS, Nantes, France ´ loic. michel@ec-nantes.

Michel in A para-model agent for dynamical systems D. p. 7
Where the paper says it

In 2019 6th International Conference on Control, Decision and Information Technologies (CoDIT), pages 1492–1497, 2019. A para-model agent for dynamical systems.

p. 7
Where the paper says it

In 2019 6th International Conference on Control, Decision and Information Technologies (CoDIT), pages 1492–1497, 2019. A para-model agent for dynamical systems.

Delen and R. p. 7
Where the paper says it

06315, 2020. Delen and R.

p. 7
Where the paper says it

06315, 2020. Delen and R.

Sharda in Artificial Neural Networks in Decision Support Systems

--- Page 1 ---

Online learning of neural networks based on a model-free control algorithm explores using a model-free-based control law to train artificial neural networks in a supervised learning context by tuning synaptic weights as a feedback control tracking problem

--- Page 2 ---

Online learning of neural networks based on a model-free control algorithm explores using a model-free-based control law to train artificial neural networks in a supervised learning context by tuning synaptic weights as a feedback control tracking problem

This paper presented an application of the model-free-based control methodology in the field of artificial neural networks Further works will include the formalization of our proposed approach as well as investigations regarding the application of our proposed algorithm to large scale neural networks The paper is organized as follows, reviewing the para-model approach in Section 2, presenting an example of distributed model-free-based control in Section 3, and presenting numerical results in Section 5 The implementation of the control scheme is depicted in Fig. 1 The evolution of the weights Wi and the controlled output y versus iterations when the network is subjected to changes xtrain1 = 0.15, xtrain2 = 0.8 at k = k1 and then ytrain = 0.6 at k = k3 as well as also subjected to a modification of the neural network topology (setting W7 = 0) at k = k2 The set of Cπ-parameters of the controller is defined as the set of the tuning coefficients Kp, Ki, kα, and kβ The control law associated to each variable xj is given by xj = Caⱼπ (yj, bj) The set of references includes G. Montavon, W. Samek, and K.-R. M¨uller in Digital Signal Processing C. C. Aggarwal in Neural Networks and Deep Learning M. Fliess and C. Join in Model-free control L. Michel in A para-model agent for dynamical systems D. Delen and R.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Model-free control for synaptic weight tuning allows artificial neural networks to be trained online by treating weight updates as a tracking control problem. This enables configuring neural networks by updating weights such that asymptotically, the output y remains 'as close as possible' to ytrain.

  2. The para-model control technique can be implemented as a derivative-free learning algorithm to tune the weights of a neural network in response to online training data. This is achieved by defining weight updates using the controller C pi: W1 = C(Kp 1,Ki 1,kα 1,kβ 1)π (y, ytrain), etc.

  3. The system can exhibit robust tracking performance even when subjected to arbitrary changes in training data and network topology. This is demonstrated by observing a great tracking of the output y has been observed despite the different changes of the training data as well as the topology of the network.

  4. The classifier example shows that a control-based training algorithm can properly classify data, where the partition of the classifier via y corresponds indeed to the 'filtered' averaged value of the output training data.

Abstract

Training artificial neural networks predominantly relies on offline, gradient-based backpropagation algorithms. In contrast, biological circuits process and retain information through decentralized, continuous-time adaptation. In this work, we explore an online, gradient-free learning methodology by formulating the tuning of synaptic weights as a closed-loop trajectory tracking problem driven by model-free control principles. Rather than performing global optimization across static parameters, decentralized controllers adjust synaptic interactions in real-time using solely local input-output tracking errors. We present an exploratory benchmark neural architecture, whose purpose is to assess the dynamic resilience of the proposed approach, particularly in tracking target outputs and adapting to abrupt non-stationarities and structural perturbations (such as topological changes or dynamic dropout) without requiring offline retraining. This empirical preliminary study provides a proof-of-concept for control-theoretic learning paradigms and outlines the foundation for scalable, dynamically evolving connectome frameworks.

Sources

Related papers