Learning reshapes power-law anisotropy in internal representations
The study investigates how internal representation geometry, characterized by power-law anisotropy, changes during neural network training. By modeling a linear network with specific input and teacher structures, researchers found that the local power-law exponent evolves nonmonotonically across different modes during feature learning. This evolution exhibits up to four distinct asymptotic regimes depending on the learning phase.