Bug or Feature squared: Weight Drift and Activation Sparsity
The study investigates how weight drift, caused by standard losses interacting with asymmetric activation functions, leads to emergent activation sparsity in neural networks. Researchers found that this drift is intrinsic to optimization and appears even with random inputs. This phenomenon suggests that the resulting sparsity can be controlled for better performance using specific techniques.