Artifacts of Numerical Integration in Learning Dynamical Systems

arXiv:2507.14491 · math.NA, cs.LG, cs.NA · Submitted 2025-07-19 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Artifacts of Numerical Integration in Learning Dynamical Systems".

Jane: The gist: The analysis demonstrates that a damped oscillatory system may be incorrectly identified as having ”anti-damping” and exhibiting a reversed oscillation direction,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: This paper dives into the learning problem, which they set up as a least squares fitting problem where you approximate the flow of a dynamical system using some numerical integrator with a step size h <ref:2507.14491#pg2>. They are looking at how reducing that step size or increasing the order of an explicit integrator doesn't fix this problem in general <ref:2507.14491#pg3>.

Jane: So, the summary points out that higher-order explicit methods have stability regions that extend further into the right half complex plane, which actually makes things worse in some cases <ref:2507.14491#pg3>. It’s not a simple matter of just using a finer step size to solve the learning artifact <ref:2507.14491#pg3>.

Lu: They are looking at two main questions: first, what happens if I use a smaller step size or higher order integrator in computing new simulations? And second, what is the generalization to initial data that is out of the original sampled distribution <ref:2507.14491#pg3>.

Meng: The paper addresses this second question by doing a local-linearization analysis through solving the optimization problem for a fixed f and y0 <ref:2507.14491#pg3>. This helps construct an approximation of the flow that is inherently constrained by the underlying numerical scheme <ref:2507.14491#pg3>.

Lalam: That constraint on the approximation, comparing it to S n h gy0, is what links the discrete simulation back to the continuous-time dynamics dy dt = g(y) <ref:2507.14491#pg3>.

The paper's summary: Tom: The authors point to a prescriptive recommendation in Section four and the conclusion <ref:2507.14491#pg1>. Since prior knowledge about the system being conservative or dissipative is often missing, they suggest using the implicit midpoint rule as a principled default choice <ref:2507.14491#pg3>.

Jane: That’s interesting because they say its stability region coincides with the left half complex plane, which preserves conservative modes exactly and extends naturally into dissipative regimes <ref:2507.14491#pg3>. It's described as "the uniquely principled choice for dynamical system learning" because of that stability region confinement <ref:2507.14491#pg3>.

Lu: They also show how the choice of integrator affects learned dynamics even when the true dynamics are damped, like in a damped Hamiltonian model where the implicit midpoint rule preserves neutral oscillation to leading order with no spurious contraction or expansion Page fifty.

Meng: In contrast, they show that the explicit Euler method biases it into the right half plane regardless of whether there is local dissipation present Page fifty. That’s a big difference in how the learning outcome looks.

Lalam: So for someone only listening to this, the main improvement is this strong recommendation: when you don't know if your system is conservative or dissipative, use the implicit midpoint rule because it naturally handles both structures Page fifty.

The paper's improvements: Tom: So to wrap up the paper "Artifacts of Numerical Integration in Learning Dynamical Systems," the big idea is that you can't just pick any integrator; it shapes what you learn about the system, and we need a principled choice for learning autonomous systems <ref:2507.14491#pg3>.

Jane: They quantify how sample size N and noise level sigma propagate into the learned parameter through perturbation analysis Page fifty-one. Theorem two point two shows that for sufficiently small noise, the first-order perturbation of the nearby minimizer is distribution-free Page fifty-one.

Lu: And they show that aggregating multiple trajectories suppresses the effect of noise, as Corollary two point two demonstrates that E

ˆqN,M (σ) − a: is O(sigma squared]

Page fifty-one: . This means more data helps clean up the noise in our learned parameter estimation.

Meng: They also identify a root-selection principle that resolves a non-uniqueness previously reported for linear multistep methods Page fifty-one. That’s useful for making sure our learning algorithms pick the right solution when things are ambiguous.

Lalam: Overall, the paper suggests that the implicit midpoint method naturally favors preserving conservative and dissipative structure from discrete data Page fifty-one. So, if you're building a learning model for an autonomous system without knowing its nature, you should use the implicit midpoint rule because it has this unique stability boundary on the imaginary axis Page fifty-one.

Tom: That’s the gist of "Artifacts of Numerical Integration in Learning Dynamical Systems." It shows that being careful about your simulation tool is just as important as having a good dataset. We'll take a quick break and then look at some of those other papers we have on our desk.

Conclusion: Tom: So we've been talking about how picking the right numerical integrator matters when you’re trying to learn the flow of a dynamical system, and this paper, "Artifacts of Numerical Integration in Learning Dynamical Systems," really hammers home that point.

Jane: Right, it shows that even if your true system is damped, using the wrong numerical method can make your AI think it's anti-damping or reversing the oscillation direction.

Lu: It’s about how the stability region of the integrator—that circle in complex plane where the method stays stable—distorts what we actually learn about that dynamics.

Meng: So, if you use an explicit method, even a higher-order one, its stability region might extend further into the right half plane, which seems to make things worse for learning compared to some implicit methods.

Lalam: The paper suggests that when you don't know if your system is conservative or dissipative, the implicit midpoint rule is the principled default because its stability region sits exactly on the left half complex plane.

Tom: Exactly, so if you're working with autonomous systems, that method naturally preserves the correct qualitative behavior—neutral oscillation when it should be neutral, and contraction when it should be contracting.

Jane: It’s also super practical because they show how aggregating multiple trajectories helps suppress noise in the learned parameter estimation by a factor of M for leading-order error Page fifty-one.

Lu: And they also identified a root-selection principle that fixes some non-uniqueness issues you might see with linear multistep methods. That’s a concrete fix for when the math gets ambiguous.

Meng: From an engineering standpoint, knowing how much noise propagates and how to aggregate data to mitigate it gives us a real handle on building more robust AI models that can handle messy real-world observations.

Lalam: And from my perspective, this work suggests that if we want AI systems to be good at understanding complex physical dynamics, we need these kinds of mechanistic priors built into the learning process.

Tom: So yeah, the title "Artifacts of Numerical Integration in Learning Dynamical Systems" really tells us that your choice of simulation tools dictates what your AI learns about a system's fundamental nature.

Jane: It’s a big reminder that we can’t just treat every numerical solver as equally good when you are trying to infer hidden physics.

Lu: We've got some other papers on the desk that look at black-box attacks and reinforcement learning policy constraints, but this one really grounds the idea of how numerical approximation affects learned dynamics.

Meng: It makes me think about how we set up our simulations for things like training those complex world models we’re building. If the underlying integrator is flawed, the whole model inherits that flaw.

Lalam: This paper provides a solid foundation for how to make our learning agents more reliable when they’re dealing with continuous physical systems.

Tom: We'll leave it there for now, but next time we’ll be looking at how AI agents actually try to trick these models, like in the Blackknife paper.

National Chung Cheng University · The University of Texas at Austin

math.NA, cs.LG, cs.NA

Submitted: 2025-07-19

Updated: 2026-10-13

DOI: 10.1007/s40687-026-00644-9

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 72/100

The gist: The gist: The analysis demonstrates that a damped oscillatory system may be incorrectly identified as having ”anti-damping” and exhibiting a reversed oscillation direction, even though it

Key concepts

Learning Problem and Artifacts: The learning problem involves finding a numerical integrator that best approximates the true system's flow. Using an integrator with step-size h creates an artifact where the resulting discrete system's dynamics are shaped by the integrator's properties, not just the true underlying physics.
Stability Region: This is a region in complex numbers that dictates which numerical methods are stable when applied to a scalar model. The learned parameter must fall within this region for the identified dynamics to be physically meaningful and preserve correct behavior.
Implicit Midpoint Rule: This specific numerical integrator is recommended as a default choice because its stability region coincides with the left half of the complex plane. This property ensures it naturally preserves conservative modes and extends correctly into dissipative regimes, unlike other methods.

Summary

The gist: The analysis demonstrates that a damped oscillatory system may be incorrectly identified as having ”anti-damping” and exhibiting a reversed oscillation direction, even though it adequately fits the given data This paper shows that the stability region of the selected integrator will distort the nature of the learned dynamics.

The Learning Problem and Artifacts

The learning problem is formulated as a least squares fitting problem where one approximates the flow of a dynamical system using a numerical integrator Specifically, the practice of approximating the propagator by using a numerical integrator with a step-size h leads to the focused least squares fitting problem. The global minimizer of this problem defines a discretetime dynamical system where yn+1:= Sh[g]yn. The analysis investigates how reducing the step-size or raising the order of an explicit integrator does not, in general, remedy this artifact because higher-order explicit methods have stability regions that extend further into the right half complex plane.

Characterization via Stability Regions

The learned parameter λhˆ must lie in the closure of the integrator’s stability region for one-step methods applied to a scalar model. The qualitative behavior of the identified dynamics is determined by the location of the method’s stability region in the complex plane. Corollary 2.1 summarizes this relationship, stating that when Re(λ) ≤ 0, and λh satisfies the Nyquist condition, then for all considered one-step methods under the Nyquist condition, the learned dynamics λˆ preserve the same rotation direction as the true conservative dynamics.

Prescriptive Recommendation

The analysis leads to a prescriptive recommendation: when prior knowledge about the underlying dynamics is only that it is autonomous, one advocates for the implicit midpoint rule as a principled default choice because its stability region coincides with the left half complex plane, preserving conservative modes exactly and extending naturally to dissipative regimes. The Implicit Midpoint Rule is noted as being the uniquely principled choice for dynamical system learning because its stability region is confined to the left half-plane.

Impact of Numerical Schemes on Dynamics

The choice of integrator significantly affects the learned dynamics, even when the true dynamics are damped. For example, in a damped Hamiltonian model, the Implicit Midpoint Rule is shown to preserve neutral oscillation to leading order with no spurious contraction or expansion. In contrast, the Explicit Euler method biases the learned spectrum into the left half plane and implicit Euler biases it into the right, regardless of whether the true local dissipation is present.

Noise and Sample Size Propagation

The analysis quantifies how sample size N and noise level σ propagate into the learned parameter through a perturbation analysis. Theorem 2.2 shows that for sufficiently small noise, the first-order perturbation of the nearby minimizer is distribution-free, while the leading-order mean-square error depends only on the noise variance σ2. Furthermore, aggregating multiple trajectories suppresses the effect of noise, as shown by Corollary 2.2 which demonstrates that E[ˆqN,M (σ) − a] = O(σ2).

Conclusion

Overall, the analysis suggests that the implicit midpoint method naturally favors the preservation of conservative and dissipative structure from discrete data. Therefore, in the absence of prior knowledge about whether an autonomous system is conservative or dissipative, one should advocate for the use of the Implicit Midpoint Rule as a principled default choice. The study also provides a way to analyze learning problems for more complex nonlinear systems by examining local linearization along sampled trajectories. The superior stability of the Implicit Euler method actually makes it the worst choice for forward computation in the type of inverse problems considered in this paper. The analysis also quantifies how sample size and observational noise propagate into the learned parameter, and identifies a root-selection principle that resolves a non-uniqueness previously reported for linear multistep methods. The implicit midpoint method is the unique method whose stability boundary coincides with the imaginary axis. The learned local spectrum sits on the correct side of the imaginary axis in both conservative and dissipative regimes, so a nonlinear learned vector field fit under the implicit midpoint rule inherits the correct qualitative local behavior.

--- Page 50 ---

How it works

The analysis focuses on the scalar linear problem defined by min λˆ∈C X N n=1 S mn h [λzˆ]z0 − z(tn; z0) 2. The results show that for one-step methods, the learned parameter λhˆ must lie in the closure of the integrator’s stability region. This is a finite-h geometric characterization of the learned dynamics, distinct from asymptotic-in-h error expansions.

Phase Errors and Rotation Direction

The imaginary part of the learned eigenvalue λhˆ preserves the sign of the ground-truth dynamics, as shown by Corollary 2.1 for all considered one-step methods under the Nyquist condition. For instance, for conservative data, Table 3 shows that Im(λhˆ) = sin θ. This implies that the rotation direction of the true dynamics is always recovered.

Noisy Data Effects

When data are noisy, Theorem 2.2 shows that E[ˆqN (σ) − a] = O(σ2). The leading-order mean-square error depends only on the noise variance σ2. Furthermore, multiple independent trajectories reduce the leading-order mean-square fluctuation by a factor of M relative to the single-trajectory case.

Nonlinear System Applications

The conclusions extend locally to nonlinear systems through linearization along any sampled trajectory γ(t). The local discrete dynamics is governed by the Jacobian of the one-step map, DSh(xn; g), which, to leading order as h → 0, is obtained by applying the chosen numerical integrator to the linearized system w˙ = Aˆ nw. The analysis shows that for conservative systems, such as the damped Hamiltonian model, the Implicit Midpoint Rule is shown to yield the most accurate learning, both in terms of amplitude and phase accuracy.

Summary of Findings

The paper concludes that the choice of integrator shapes the learned dynamics even in the absence of structural priors. The Implicit Midpoint Rule is advocated as a principled default choice because its stability region coincides with the left-hand side of the complex plane. The analysis also quantifies how sample size and observational noise propagate into the learned parameter, and identifies a root-selection principle that resolves a non-uniqueness previously reported for linear multistep methods.

--- Page 51 ---

How it works

Improvements for AI systems

Bold Header: Prescriptive Integrator Selection for Autonomous Systems

By advocating for the implicit midpoint rule as a principled default choice, since its stability region coincides with the left half complex plane, it preserves conservative modes exactly, and extends naturally to the dissipative regime, AI systems can reliably infer the correct qualitative local behavior—neutral oscillation where the truth is conservative, contraction where the truth is dissipative.

Bold Header: Artifact Mitigation via Data Aggregation

The analysis shows that aggregating multiple trajectories suppresses the effect of noise, meaning AI systems using methods like those in Corollary 2.2 can achieve an error reduction where the leading-order mean-square fluctuation is reduced by a factor of M relative to the single-trajectory case.

Bold Header: Robust Eigenvalue Identification for Linear Systems

For linear systems, the learned parameter λˆ must lie in the closure of the integrator’s stability region, allowing AI to distinguish between methods such that higher-order explicit methods, whose stability regions extend further into the right half-plane, may in fact worsen rather than reduce this artifact.

Abstract

In many applications, one needs to learn a dynamical system from its solutions sampled at a finite number of time points. The learning problem is often formulated as an optimization problem over a chosen function class. However, in the optimization procedure, prediction data from generic dynamics requires a numerical integrator to assess the mismatch with the observed data. This paper reveals potentially serious effects of a chosen numerical scheme on the learning outcome. Specifically, the analysis demonstrates that a damped oscillatory system may be incorrectly identified as having "anti-damping" and exhibiting a reversed oscillation direction, even though it adequately fits the given data points. This paper shows that the stability region of the selected integrator will distort the nature of the learned dynamics. Crucially, reducing the step size or raising the order of an explicit integrator does not, in general, remedy this artifact, because higher-order explicit methods have stability regions that extend further into the right half complex plane. Furthermore, it is shown that the implicit midpoint method can preserve either conservative or dissipative properties from discrete data, offering a principled integrator choice even when the only prior knowledge is that the system is autonomous.