Spectrum Matching: a Unified Perspective for Superior Diffusability in Latent Diffusion
The work introduces Spectrum Matching, a hypothesis suggesting that latents with better diffusion properties must follow specific spectral patterns: Encoding Spectrum Matching (ESM) and Decoding Spectrum Matching (DSM). By enforcing these spectral alignments using regularization losses, the authors show that VAEs can inherit superior generation qualities from pixel diffusion models. The findings suggest matching the latent spectrum to target spectra improves semantic content generation in latent diffusion.