Dimensionless Controls of Plasticity Under Alternating Tasks: From Evolutionary Biology to Continual Learning
math.OC, cs.LG
Submitted: 2026-08-24
Updated: 2026-08-24
Comments: 21 pages, 5 figures. Published in PMLR 334(2):87-107, 2026
Journal ref: Proceedings of the 2nd Conference on Topology, Algebra, and Geometry in Data Science (TAG-DS 2026), PMLR 334(2):87-107, 2026
License: http://creativecommons.org/licenses/by/4.0/
The gist: Plasticity under changing environments is central to both evolutionary biology and continual learning.
Terminology
Abstract
Plasticity under changing environments is central to both evolutionary biology and continual learning. Motivated by recent work on genotype--phenotype maps, we study a minimal deep-learning analogue where a network is trained alternately on two Boolean label sets, and ask which biological controls of plasticity survive the translation to gradient descent. Reinterpreting four proposed biological factors as quantities of training dynamics, we find the system reduces to two dimensionless controls: the task disagreement r, the fraction of disagreeing labels, and the reach ηT, the product of learning rate and switching period. We derive two bounds on plasticity: r alone fixes an extremal geometric floor on the utopia distance, while r and ηT jointly bound forgetting. Across 9,720 trajectories, an ANOVA confirms that r, η, and T dominate, while the effect of neutral-set size (emphasized in the biological setting) is negligible. The optimal reach itself follows an approximate inverse power law ηT* proportional to r-1.18, yielding a heuristic that sets the optimal reach ηT* from the task disagreement alone. The analogy that survives is therefore dynamical rather than geometric, and our setting enables a view of plasticity through the lens of other driven systems in physics and engineering.
Sources
- Disentangling Linear Mode-Connectivity
- The Loss Surfaces of Multilayer Networks
- Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability
- Deep Ensembles: A Loss Landscape Perspective
- Out-of-distribution forgetting: vulnerability of continual learning to intra-class distribution shift
- The Platonic Representation Hypothesis
- Deep linear neural networks with arbitrary loss: All local minima are global
- Linear Mode Connectivity in Multitask and Continual Learning
- On the curvature of the loss landscape
- Experience Replay for Continual Learning
- There Will Be a Scientific Theory of Deep Learning
- When Representations Align: Universality in Representation Learning Dynamics
- Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer
- Gradient Surgery for Multi-Task Learning
- Why flatness does and does not correlate with generalization for deep neural networks
- Non-equilibrium physics: from spin glasses to machine and neural learning
Related papers
- Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
- Adam-HNAG: A Convergent Reformulation of Adam with Accelerated Rate
- Incremental Learning in Mirror Flows
- Beyond Aggregate Drift: Optimal Client-Specific Windows in Federated Learning
- Lower Bounds for Randomized Linear-Oracle Optimization over Strongly Convex Sets
- Second-Order Optimization with Asynchronous Curvature