Bayes-optimal learning of an extensive-width neural network from quadratically many samples
The discussion of 'Bayes-optimal learning of an extensive-width neural network from quadratically many samples' presents a theoretical benchmark for optimal network learning. The hosts explore how a practical algorithm, GAMP-RIE, achieves this limit. A key finding is that averaging multiple runs of gradient descent can reach this optimal error in noiseless scenarios.