DraftAttention2: Fast Video Diffusion with Low-Resolution-Guided Mixed-Precision Attention
cs.CV, cs.AI
Submitted: 2026-09-26
Updated: 2026-09-26
Comments: Preprint Version
Code: https://github.com/anemoi-project/anemoi
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Video generation has broad applications in content creation and entertainment.
Terminology
Abstract
Video generation has broad applications in content creation and entertainment. Diffusion transformers have advanced the quality of generated videos, but attention over spatiotemporal tokens becomes increasingly expensive as video resolution and duration increase. We present DraftAttention2, a training-free framework that uses the low-resolution draft attention map to jointly select attention blocks and assign their numerical precision. Specifically, spatial 2D average- and max-pooled queries and keys capture complementary regional statistics to estimate block importance, and a shared ranking assigns higher precision to important blocks, lower precision to less important retained blocks, and skips the rest under configurable budgets. Our analysis separates sparsification error from attention-weighted quantization error, establishing when recovering skipped interactions with low-bit computation tightens the output-error bound. This analysis motivates retaining more interactions at low precision while reserving higher precision for blocks with larger attention mass. To translate these fine-grained assignments into practical speedups, we further develop fused operand preparation and a single attention kernel with precision-specific phases, sharing data movement, softmax statistics, and output accumulation across precisions. Experiments demonstrate that our method achieves a superior quality-efficiency trade-off over existing efficient video generation methods. Notably, its advantage is particularly pronounced for few-step video diffusion, where jointly combining sparsity with 4- and 8-bit mixed-precision computation substantially improves generation quality while retaining significant acceleration. Code is available at https://github.com/anemoi-project/anemoi
Sources
- HunyuanVideo: A Systematic Framework For Large Video Generative Models
- Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention Sparsification
- Attention Sparsity is Input-Stable: Training-Free Sparse Attention for Video Generation via Offline Sparsity Profiling and Online QK Co-Clustering
- ThriftAttention: Selective Mixed Precision for Long-Context FP4 Attention
- DraftAttention: Fast Video Diffusion via Low-Resolution Attention Guidance
- Wan: Open and Advanced Large-Scale Video Generative Models
- XAttention: Block Sparse Attention with Antidiagonal Scoring
- SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routing
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models