A Theoretical Framework for Statistical Evaluability of Generative Models
This episode discusses a paper providing a theoretical framework for statistically evaluating generative models using finite data. Hosts conclude that not all metrics are reliable; some, like Rényi divergences, are mathematically impossible to estimate from limited samples. Perplexity can only be trusted if the model's probability is close to the ground truth.