Dynamic gain neuromodulation attenuates stability gaps during continual joint training
The paper introduces Dynamic Gain Neuromodulation (NGM-SGD) to reduce stability gaps in continual learning by using an uncertainty-modulated gain mechanism. This mechanism transiently increases neuronal gain during task transitions to reshape optimization trajectories, balancing rapid adaptation with stable knowledge retention. Results show NGM-SGD outperforms existing methods like MSGD and Adam on benchmarks like MNIST and CIFAR.