Fine-Grained Caching for Diffusion Transformers with Few Calibration Conditions
InvarDiff creates a training-free caching scheme for diffusion models by exploiting feature invariance across different timestepscales and layer scales. It computes a deterministic cache plan based on feature change statistics, allowing it to reuse computations across steps and layers. This results in 2–3x speedups with minimal quality loss.