Imaginative Generative AI: Crossing the Entropy Wall into Worlds Beyond Imitation
The episode discusses a paper from Chinese University of Hong Kong proposing that generative AI should target diversity, not just imitation. They introduce an 'entropy wall'—the spectral entropy of real data—and show how pushing beyond it enables controlled extrapolation. They demonstrate inference-time guidance for diffusion models, producing novel outputs like transformed skyscrapers and thrones.