Prediction of chaotic dynamics from data: An introduction
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Nonlinear Sciences. The day in nonlinear science: chaos, patterns, networks and adaptation.
Kai: We're looking at the paper: "Prediction of chaotic dynamics from data: An introduction".
Mira: Here's the gist of the paper. Prediction of chaotic dynamics from data: An introduction offers a principled approach to predicting chaotic systems from data by introducing concepts from dynamical systems theory and machine learning approaches like echo state networks and long-short-term memory networks The chapter introduces deterministic systems, chaos, and machine learning methods for time-forecasting chaotic dynamics Chaotic Dynamical Systems Concepts Deterministic systems are noisefree systems where there exists one solution that corresponds to an initial condition Chaos is a deterministic phenomenon characterized by erratic behaviour that is difficult—yet possible, in principle—to predict Chaotic dynamics are characterized by the extreme sensitivity to small perturbations, for example, in the initial conditions, or the parameters, or external forcing The long-term statistics of turbulent flows may be more predictable than the instantaneous time dynamics Lyapunov Exponents and Chaos Metrics The largest (dominant) Lyapunov exponent, λ1, is the time-averaged growth rate of infinitesimal perturbations The dominant Lyapunov exponent indicates the type of solution (i.e., attractor), where λ1 < 0 means perturbations decay and the attractor is a fixed point, while λ1 = 0 suggests a periodic orbit or chaos The Lyapunov spectrum provides N Lyapunov exponents, which are the eigenvalues of the Oseledets matrix The Kaplan-Yorke conjecture proposes an estimate (upper bound) of the attractor’s dimension as DKY = k + Pk i=1 λi λk+1 Machine Learning Architectures for Dynamics Recurrent Neural Networks (RNNs) are dynamical systems because the connection within RNN have cycles enabling a self-sustained temporal activation dynamics and a dynamical memory of the input excitation Echo State Networks (ESNs) are a form of reservoir computing where the outputs are obtained by a linear combination of the reservoir states driven by inputs Long Short-Term Memory (LSTM) networks feature a cell state responsible for retaining long-term information and a hidden state focused on capturing short-term memory Physics-Informed Modeling and Training The Physics-informed ESN (PI-ESN) incorporates prior knowledge by combining the Mean Squared Error (MSE) with a physical residual term The PI-LSTM uses a loss function L(x, ˆx) = Ldd(x, ˆx) + αpiLpi(ˆx), where Ldd is the data-driven loss and Lpi is the physics-informed loss The PI-LSTM can enforce governing equations by minimizing the residual R(x(ti+1)) = x(ti+1) − x(ti) + Z ti+1 t0 F (x(t)) dt Validation Strategies for Chaotic Models Robust validation strategies include Single Shot Validation (SSV), Walk Forward Validation (WFV), K-fold cross validation (KFV), and Recycle Validation (RV) The chaotic version of a validation strategy consists of shifting the validation intervals forward in time by one Lyapunov time to further increase the number of intervals on which the network is validated Performance Assessment Metrics The predictability is key, defined as the Lyapunov time, which is the inverse of the largest Lyapunov exponent tp ≡ 1/λ1 The prediction horizon (PH) is a commonly used metric tailored for the prediction of diverging trajectories in chaotic dynamics The Lyapunov exponents serve as indicators of the network’s accuracy in inferring stability properties and its ability to reproduce the chaotic nature in the prediction LSTM and ESN Performance Comparison For evaluation, both networks are employed in closed-loop mode on unseen test data, where they compare based on short-term prediction capabilities The average prediction horizon for the ESN is 3.16tp while for the LSTM it is 5.27tp when tested over M intervals of 10 Both networks can be employed to analyze further stability properties, such as covariant Lyapunov vectors The inferred Lyapunov exponents for the LSTM and ESN are 0.9067 and 0.
Title and authors: Kai: So we've talked about what chaos is and the tools used to measure it, which leads us into how machine learning fits in. The paper introduces recurrent neural networks as a dynamical system because their internal connections create cycles that allow for self sustained temporal activation dynamics and a kind of memory of what came before.
Mira: And then they bring in echo state networks as a form of reservoir computing where the outputs are determined by taking a linear combination of the states in that reservoir driven by our inputs. That’s how we start mapping these complex dynamics onto neural network structures.
Lev: For us, the immediate question is what kind of training signals we need for an echo state network to actually capture those chaotic features instead of just learning some simple pattern.
Kai: The paper also discusses long short term memory networks, or LSTMs, which have a special cell state that’s designed to retain information over long periods and a hidden state that focuses on capturing shorter term memory.
Mira: And they suggest combining these ideas with physics-informed modeling, specifically using the mean squared error loss combined with a physical residual term. This is where things get interesting for us theorists.
Lev: That physics informed approach lets you inject prior knowledge into the learning process by minimizing a loss function that includes both what the data shows and what we know about the governing equations of the system.
Kai: For example, they use a loss function where one part is data driven, and another part is physics informed, which helps constrain the network to actually follow those underlying physical rules.
Mira: It’s a way to handle situations where you might only have partial observations of the system, because the physical constraints help guide the learning when data alone isn't sufficient.
The paper's summary: Kai: So to summarize what this paper is proposing, it’s that we can use these machine learning approaches, specifically echo state networks and LSTMs, not just as pattern recognizers but as genuine dynamical systems models for forecasting chaotic dynamics.
Mira: They are moving away from just looking at the numbers and instead embedding the mathematical structure of dynamical systems directly into the learning architecture itself. It’s about using ML to understand the dynamics, not just map them.
Lev: From an engineering standpoint, if we use these models for forecasting, we need to be careful about how much data they actually need before they start producing reliable predictions for diverging trajectories.
Kai: The paper suggests that while instantaneous prediction is tough because of chaos, the long term statistics of turbulent flows might be more predictable even with these methods.
Mira: They are using validation strategies like single shot validation and walk forward validation, but they introduce a chaotic version of those by shifting the validation intervals forward by one Lyapunov time to test robustness.
Lev: Testing against a Lyapunov time shift is clever because it directly tests how well the network handles the inherent instability of chaotic systems over longer prediction horizons.
Kai: The paper also focuses heavily on performance assessment, defining predictability through the Lyapunov time, which is simply the inverse of that largest Lyapunov exponent.
Mira: They use a metric called the prediction horizon, or PH, which is tailored specifically for predicting diverging trajectories in chaotic dynamics.
The paper's improvements: Kai: Now let’s talk about what they suggested improving this approach to make it more robust. One major suggestion is to incorporate physics informed loss functions directly into echo state networks to constrain the learned dynamics using governing equations.
Mira: That makes sense, because if you only have limited data, the network needs that physical guidance to avoid learning something physically nonsensical. It helps keep the network grounded in reality.
Lev: From an error-correction perspective, this constraint is vital for real hardware; it prevents the learned dynamics from drifting into states that would destroy a qubit or whatever physical system you’re using.
Kai: Another improvement involves hybrid loss functions, combining data driven loss with a physics informed loss term that incorporates the integral formulation of the dynamical system constraints.
Mira: That combination is powerful because it forces the network to satisfy both what we observe in our measurements and what the actual mathematical equations require over time.
Lev: That's how you get reliable long-term behavior, because you’re not just fitting a curve to a few points; you’re ensuring the curve respects the physics of how that system evolves.
Kai: They also mentioned using techniques like walk forward validation and recycle validation specifically for tuning hyperparameters, which is crucial when dealing with chaotic time series.
Mira: Tuning parameters for chaotic systems is tricky because the optimal setting might depend on where you are in the data sequence, so those sequential validation methods help select better settings.
Conclusion: Kai: So wrapping up "Prediction of chaotic dynamics from data: An introduction", the core idea is using echo state networks and LSTMs to model chaotic systems by grounding them in dynamical systems theory, incorporating physics constraints through hybrid loss functions for better learning.
Mira: And they emphasize that while instantaneous prediction is hard due to chaos, we can gain insight into the long term statistics of turbulent flows if we use appropriate validation strategies like walk forward validation.
Lev: For me, the implication is that this framework gives us a principled way to see what kind of prediction horizon we should actually expect when deploying this on real hardware, because it’s tied directly to those Lyapunov exponents.
Kai: It moves the discussion from just building a model that fits data to building a model that understands the physics of how chaos behaves, which is what this paper aims for.
Mira: It’s an introduction to using ML not just as a black box but as a tool constrained by known physical laws. That's where the value lies.
Lev: We’ll keep looking at how we can use these concepts to build error correction codes that respect these dynamics, because that's where the real engineering challenge is.
Imperial College London · Alan Turing Institute · Politecnico di Torino · University of Cambridge
nlin.CD
Submitted: 2026-04-13
Updated: 2026-04-13
Comments: In: Machine Learning for Fluid Mechanics, ed. by M. A. Mendez and A. Parente. von Karman Institute for Fluid Dynamics, 2025
Code: https://github.com/MagriLab/Tutorials
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 88/100
The gist: Prediction of chaotic dynamics from data: An introduction offers a principled approach to predicting chaotic systems from data by introducing concepts from dynamical systems theory and machine…
Key concepts
- Deterministic Systems
- These are noise-free systems where a single starting point uniquely determines the entire future trajectory. Chaos is a specific type of deterministic system that shows erratic behavior, making long-term prediction extremely difficult despite being fundamentally predictable.
- Lyapunov Exponents
- This metric measures how fast small differences in initial conditions grow over time. The largest exponent ($\lambda_1$) is crucial; if it's negative, the system settles to a fixed point, but if it's zero or positive, it indicates periodic orbits or chaos.
- Recurrent Neural Networks (RNN)
- RNNs are used because their internal connections create cycles that allow them to maintain temporal memory. This structure enables them to model dynamical systems by having a self-sustained activation pattern based on past inputs and states.
- Long Short-Term Memory (LSTM) Networks
- LSTMs are specialized RNNs designed for time series data. They use a cell state to store long-term information and a hidden state to capture short-term dependencies, making them effective for modeling complex temporal dynamics.
Terminology
Summary
Prediction of chaotic dynamics from data: An introduction offers a principled approach to predicting chaotic systems from data by introducing concepts from dynamical systems theory and machine learning approaches like echo state networks and long-short-term memory networks The chapter introduces deterministic systems, chaos, and machine learning methods for time-forecasting chaotic dynamics
Chaotic Dynamical Systems Concepts
Deterministic systems are noisefree systems where there exists one solution that corresponds to an initial condition Chaos is a deterministic phenomenon characterized by erratic behaviour that is difficult—yet possible, in principle—to predict Chaotic dynamics are characterized by the extreme sensitivity to small perturbations, for example, in the initial conditions, or the parameters, or external forcing The long-term statistics of turbulent flows may be more predictable than the instantaneous time dynamics
Lyapunov Exponents and Chaos Metrics
The largest (dominant) Lyapunov exponent, λ1, is the time-averaged growth rate of infinitesimal perturbations The dominant Lyapunov exponent indicates the type of solution (i.e., attractor), where λ1 < 0 means perturbations decay and the attractor is a fixed point, while λ1 = 0 suggests a periodic orbit or chaos The Lyapunov spectrum provides N Lyapunov exponents, which are the eigenvalues of the Oseledets matrix The Kaplan-Yorke conjecture proposes an estimate (upper bound) of the attractor’s dimension as DKY = k + Pk i=1 λi λk+1
Machine Learning Architectures for Dynamics
Recurrent Neural Networks (RNNs) are dynamical systems because the connection within RNN have cycles enabling a self-sustained temporal activation dynamics and a dynamical memory of the input excitation Echo State Networks (ESNs) are a form of reservoir computing where the outputs are obtained by a linear combination of the reservoir states driven by inputs Long Short-Term Memory (LSTM) networks feature a cell state responsible for retaining long-term information and a hidden state focused on capturing short-term memory
Physics-Informed Modeling and Training
The Physics-informed ESN (PI-ESN) incorporates prior knowledge by combining the Mean Squared Error (MSE) with a physical residual term The PI-LSTM uses a loss function L(x, ˆx) = Ldd(x, ˆx) + αpiLpi(ˆx), where Ldd is the data-driven loss and Lpi is the physics-informed loss The PI-LSTM can enforce governing equations by minimizing the residual R(x(ti+1)) = x(ti+1) − x(ti) + Z ti+1 t0 F (x(t)) dt
Validation Strategies for Chaotic Models
Robust validation strategies include Single Shot Validation (SSV), Walk Forward Validation (WFV), K-fold cross validation (KFV), and Recycle Validation (RV) The chaotic version of a validation strategy consists of shifting the validation intervals forward in time by one Lyapunov time to further increase the number of intervals on which the network is validated
Performance Assessment Metrics
The predictability is key, defined as the Lyapunov time, which is the inverse of the largest Lyapunov exponent tp ≡ 1/λ1 The prediction horizon (PH) is a commonly used metric tailored for the prediction of diverging trajectories in chaotic dynamics The Lyapunov exponents serve as indicators of the network’s accuracy in inferring stability properties and its ability to reproduce the chaotic nature in the prediction
LSTM and ESN Performance Comparison
For evaluation, both networks are employed in closed-loop mode on unseen test data, where they compare based on short-term prediction capabilities The average prediction horizon for the ESN is 3.16tp while for the LSTM it is 5. p. 127tp when tested over M intervals of 10 Both networks can be employed to analyze further stability properties, such as covariant Lyapunov vectors The inferred Lyapunov exponents for the LSTM and ESN are 0. p. 99067 and 0.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Implement physics-informed loss functions in Echo State Networks to constrain learned dynamics using governing equations, such as minimizing a loss function Lphys that includes both data-driven MSE and a physical residual Lp. This allows the network to be constrained to fulfil the governing equations, which is particularly advantageous when only partial observations of the system are available.
-
Integrate validation strategies like Walk Forward Validation (WFV) and Recycle Validation (RV) for training hyperparameters, as these methods are robust for chaotic time series prediction by partitioning data sequentially or reusing data across open-loop and closed-loop configurations to select optimal network parameters.
-
Utilize the Jacobian of the learned model, such as JacESN or JacLSTM, to compute Lyapunov exponents directly from the network's internal states during closed-loop operation. This allows for an assessment of the network's accuracy in inferring stability properties and its ability to reproduce chaotic nature in predictions.
-
Employ a hybrid loss function L(x, x hat) that combines data-driven loss Ldd(x, x hat) and a physics-informed loss αpiLpi(x hat). This combination constrains the network using both observed data and physical constraints derived from the integral formulation of the dynamical system.
-
Utilize LSTM architectures with sliding windows for sequential data preparation, carefully selecting window size in consideration of hidden state dimension to balance learning short-term and long-term temporal dependencies effectively.
Related papers
- PRISM: Unifying multiphysics optothermal transport and identifiability-guided inference for nanoscale thermal metrology
- Quantifying The Complex Spatiotemporal Chaos of Cardiac Fibrillation in Ionic Models Across Parameter Regimes
- Modal interactions in strongly nonlinear coupled oscillators with closely spaced modes and nonclassical damping distribution
- An active Lorentz gas: walking droplets in periodic media
- Stochastic-layer width as an order parameter for the integrability-to-chaos transition in the bouncer model
- When more data become less informative: Finite-precision periodicization and collapse of forecast-error Lyapunov estimates