Force without transmission: a depth-induced rank collapse that no loss on the representation reopens
The study investigated whether corrective forces (penalties or hinges) could fix a 'rank collapse' in deep neural networks caused by weakening skip connections during training. The research found that these forces failed because their gradients never reached the necessary weights to change, meaning restoring the skip connection was the only way to recover network rank.