Continual Learning via Self-Probe Gradients
cs.LG, cs.AI
Submitted: 2026-09-26
Updated: 2026-09-26
Comments: 27 pages, 18 figures
Code: https://github.com/umamicode/cplus-continual
License: http://creativecommons.org/licenses/by/4.0/
The gist: Adapting pretrained models to new data can cause catastrophic forgetting of previously learned behavior.
Terminology
Abstract
Adapting pretrained models to new data can cause catastrophic forgetting of previously learned behavior. When only a few past samples remain, they give continual learning methods sparse and narrow evidence about what to preserve. We show that language models can expand this evidence through self-probing, in which the frozen model generates new inputs from the retained samples and records its own predictions on them. Unlike prior work that replays such data as training examples, our method, CPLUS uses self-probe and past-sample gradients to scale down parameter updates that conflict with prior behavior. Experiments with five language models on four benchmarks show three results. First, the same probes preserve more prior behavior as gradient signals than as replay data. Second, CPLUS learns the new data while consistently reducing forgetting more than existing baselines, especially when past data are scarce, and this protection extends to benchmarks not used for training. Third, we observe that CPLUS also becomes more effective as models grow: within the Qwen3 model family, it recovers an increasing share of the forgetting caused by standard fine-tuning.
Sources
- Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
- Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection
- Language Models are Few-Shot Learners
- Rebalancing Batch Normalization for Exemplar-based Class-Incremental Learning
- Efficient Lifelong Learning with A-GEM
- Forget Forgetting: Continual Learning in a World of Abundant Memory
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Orthogonal Gradient Descent for Continual Learning
- Measuring Massive Multitask Language Understanding
- Distilling the Knowledge in a Neural Network
- Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal
- What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams
- Continual Learning on Noisy Data Streams via Self-Purified Replay
- Overcoming catastrophic forgetting in neural networks
- Optimal Continual Learning has Perfect Memory and is NP-hard
- Replaying pre-training data improves fine-tuning
- Continual evaluation for lifelong learning: Identifying the stability gap
- Forgetting in Language Models: Capacity, Optimization, and Self-Generated Replay
- iCaRL: Incremental Classifier and Representation Learning
- Self-generated Replay Memories for Continual Neural Machine Translation
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks