From Order to Distribution: An Exact Operator Framework for Forgetting in Continual Learning

arXiv:2604.13460 · cs.LG, cs.AI · Submitted 2026-04-15 · Read on arXiv

cs.LG, cs.AI

Submitted: 2026-04-15

Updated: 2026-10-01

Terminology

• Continual Learning — 2× in this paper · explained in 2 episodes
• Adaptation — 1× in this paper · explained in 1 episode

Sources

• Theoretical Insights into Overparameterized Models in Multi-Task and Replay-Based Continual Learning
• Dark Experience for General Continual Learning: a Strong, Simple Baseline
• Last Iterate Convergence of Incremental Methods and Applications in Continual Learning
• Analysis of Overparameterization in Continual Learning under a Linear Model
• The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting -- An Analytical Model
• Preasymptotic Convergence of Randomized Kaczmarz Method
• Convergence and Implicit Bias of Gradient Descent on Continual Linear Classification
• Optimal Rates in Continual Linear Regression via Increasing Regularization
• Replay Can Provably Increase Forgetting
• Convergence of Continual Learning in Homogeneous Deep Networks
• Randomized Kaczmarz converges along small singular vectors
• Are Greedy Task Orderings Better Than Random in Continual Linear Regression?
• Optimal Training-Time Scaling in Gradual Adaptation
• A Statistical Theory of Regularization-Based Continual Learning

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks