Understanding Latent-Dimension Scaling in Dynamical-System Learning through Spectral Reliability

The gist: The spectral-residual model reduces median windowed rollout error more consistently with increasing latent dimension and by a larger factor from the smallest to the largest dimension than the latent-prediction model.

Model Comparison and Performance

The study compares two models of a shared Koopman autoencoder trained alternately for reconstruction and latent evolution, using either latent-prediction loss or spectral-residual loss. Across six chaotic systems, both models reduced median windowed rollout error from smallest to largest dimension. The spectral-residual model achieved lower medians than the latent-prediction model for all systems and dimensions. Its median fell by a larger factor in every system. Furthermore, its median decreased monotonically with dimension in four systems, against one for latent prediction. Against four baseline families, its mean valid prediction times were nearly always longer.

Spectral Reliability and Convergence

The spectral-residual loss is defined by the empirical relative residuals of candidate eigenpairs. The hypothesis is that Koopman spectral reliability helps explain how consistently rollout error decreases with increasing dimension. The paper establishes that for bounded operators and sequences of learned spaces that asymptotically approximate every observable in L2, the minimal residual converges pointwise to the infimum of the relative residual over the whole observable space. Under two-stage training at the largest dimension in each of the six chaotic systems, estimated Koopman eigenvalues of the spectral-residual model concentrated in low-residual regions for its learned dictionaries. This concentration of eigenvalues corresponds to longer mean VPT.

Experimental Setup and Metrics

The evaluation involves comparing the two models across eight variants and four baseline families using six chaotic dynamical systems. The performance is quantified by the Variance-Scaled Root Mean Squared Error (VRMSE) and Valid Prediction Time (VPT). The comparison is conducted across latent dimensions N ∈ N0, 2N0, 4N0, 8N0, and 16N0 relative to each system’s input dimension N0.

Theoretical Analysis of Latent Scaling

The theoretical analysis shows that as the space spanned by the learned dictionary approximates the observable space, the minimal residual over the dictionary space converges to the infimum of the relative residual over the whole observable space. This result provides a mathematical basis for expecting residual-based spectral information to become more reliable as the latent dimension increases. The convergence analysis shows that for every g ∈ H and δ > 0, there exists N∗ ∈ N such that, for every N ≥ N∗, there exists vN ∈ VN satisfying∥g − vN∥H +∥Kg − KvN∥H < δ.

Conclusion on Spectral Reliability

Under two-stage training at the largest evaluated dimension, the spectral-residual model achieved lower median windowed VRMSE than the latent-prediction model under two-stage training and under joint training with the operatornorm penalty in five systems. The concentration of eigenvalues of the spectral-residual model in low-residual regions for its learned dictionary corresponds to longer mean VPT. Predictions from the spectral-residual model preserved the overall shapes of several reference attractors, whereas in field predictions for Lorenz-96 and Kuramoto–Sivashinsky, structures were distorted for both models despite the concentration of spectral-residual eigenvalues. The next question is how spectral residuals, the propagation of one-step prediction errors in latent coordinates, and decoding jointly determine the dependence of rollout error on latent dimension.

The work was supported in part by JSPS KAKENHI Grant Numbers JP24K20864 and JST Moonshot R&D Program Grant Number JPMJMS263K from the Japan Science and Technology Agency. The authors have no relevant financial or nonfinancial interests to disclose. All experiments ran on NVIDIA RTX 6000 Ada graphics processing units (GPUs). This work was supported in part by JSPS KAKENHI Grant Numbers JP24K20864 and JST Moonshot R&D Program Grant Number JPMJMS263K from the Japan Science and Technology Agency. The code and configuration files used to generate the data and the numerical results included in this paper are available at https://github.com/uosaka-mlsyslab/koopman-autoencoder-spectral-reliability.

1. Bold representation of spectral reliability in model training: Implement a training objective using spectral-residual loss (relative residuals of candidate eigenpairs) instead of latent-prediction loss to learn latent evolution, as this model achieved lower medians than the latent-prediction model for all systems and dimensions.

2. Enhancement of dimension scaling consistency: Leverage the finding that the spectral-residual model is more robust to dimension increases by noting that it reduced median windowed rollout error more consistently with increasing latent dimension and by a larger factor from the smallest to the largest dimension than the latent-prediction model.

3. Improved spectral reliability for chaotic systems: Utilize spectral-residual eigenvalues concentrated in low-residual regions at the largest dimensions, as this is consistent with the hypothesis that Koopman spectral reliability explains how consistently rollout error decreases with increasing dimension.

4. Enhanced prediction accuracy via spectral information: Use the learned dictionary to estimate the minimal residual over VN by the following expression: res cM(λ, g) ≥ 0 computed from snapshot data as the nonnegative square root of the following expression [Eq. (4)] to guide prediction, as Operationally, spectral reliability means that the eigenvalues of Ab fall where τb num M,N is small.

5. Increased valid prediction time: Employ the spectral-residual model in deployment, as it achieved a longer mean VPT than the latent-prediction model at all five evaluated latent dimensions for each of the six systems, suggesting better long-term forecast stability.