Mitigating Memorization Where It Happens
Large language models memorize training data verbatim, causing privacy and generalization issues. The authors developed Gated Subspace Steering (GSS), a method that selectively intervenes only at token positions showing memorization signals. GSS uses an optimal probe-steer framework derived from matrix optimization to detect and correct memorization with minimal computational overhead, outperforming existing methods.