Looped Diffusion Transformer
Researchers explored scaling text-to-image models by repeatedly running shared Transformer blocks within each denoising step, increasing computational depth without adding parameters. They found that naive looping failed due to information loss. The solution involved deep supervision across all loop depths and self-modulating attention mechanisms, leading to models that outperform larger counterparts with lower compute.