In Two Minds about Lifelong Learning: Exploring Hemispheric Redundancy and Specialisation in Neural Models
cs.LG, cs.AI
Submitted: 2026-08-20
Updated: 2026-09-27
Comments: 21 pages before references/appendix, 9 figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: Persistent intelligent systems require the ability to learn continually, but current machine learning approaches face significant challenges in this area compared to biological learning systems.
Terminology
Abstract
Persistent intelligent systems require the ability to learn continually, but current machine learning approaches face significant challenges in this area compared to biological learning systems. Machine learning algorithms typically trade off retention of previously learned information and adaptation to new or changing data patterns. When continual learning capabilities are absent, algorithms must undergo retraining using the entire data set, an approach that becomes impractical when original training data are unavailable due to storage constraints, financial or computational costs, or privacy restrictions. However, biological animals can learn continually, without experiencing catastrophic forgetting. This paper attempts to build a high-level framework for how animals learn and preserve knowledge by modelling neural components and states that are known to be related to memory consolidation. We focus on three concepts: experience replay, REM sleep, and bilaterality. We propose 4MAS (4 Module Awake/Sleep), a novel macroarchitecture demonstrating how machine learning models might benefit from asymmetric hemispheres, each with their own long- and short-term memory mechanisms, and how a period of sleep between incremental learning tasks might benefit memory consolidation. Finally, we present results showing that our architecture achieves statistical improvements over established generative replay baselines on Split-MNIST (98.3%) and Split-Fashion-MNIST (84.9%), while reducing forgetting by more than half relative to the strongest replay baseline and demonstrating monotonic capacity scaling on Split-CIFAR-100 (29.29%).
Sources
- Measuring Catastrophic Forgetting in Neural Networks
- Three scenarios for continual learning
- A continual learning survey: Defying forgetting in classification tasks
- CNN Features off-the-shelf: an Astounding Baseline for Recognition
- DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
- Progressive Neural Networks
- Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
- Learning without Isolation: Pathway Protection for Continual Learning
- Load Balancing Mixture of Experts with Similarity Preserving Routers
- The Effectiveness of Memory Replay in Large Scale Continual Learning
- Continual Learning with Deep Generative Replay
- Generative Feature Replay For Class-Incremental Learning
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- SpaceNet: Make Free Space For Continual Learning
- Joint Diffusion models in Continual Learning
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks