Equivalent Flows, Unequal Learning: Clean-Latent Prediction in Transformers
This study compared two prediction methods in latent diffusion models: predicting a clean latent point versus predicting a matched velocity. The findings show that clean-latent prediction consistently outperforms velocity regression because of geometric differences between the target distributions, specifically how they handle low-variance directions. This suggests that choosing the correct target geometry is crucial for effective denoising.