Focused Forcing: Content-Aware Per-Frame KV Selection for Efficient Autoregressive Video Diffusion
cs.CV, cs.AI
Submitted: 2026-05-18
Updated: 2026-10-29
License: http://creativecommons.org/licenses/by/4.0/
The gist: Recent advances in autoregressive video diffusion have enabled sequential and streaming video generation.
Terminology
Abstract
Recent advances in autoregressive video diffusion have enabled sequential and streaming video generation. However, long-horizon generation requires increasingly large KV caches, making efficient compression without sacrificing quality challenging. Existing methods mostly select historical frames based on attention scores, but their context decisions remain coarse. When multiple frames are generated in the same chunk, these methods often apply a shared history selection to the whole chunk, score historical frames solely by attention, and assign head-wise budgets either uniformly or by attention-pattern heuristics rather than explicit head-importance estimation. We show that frames within the same generated chunk can depend on distinct historical frames, that the same historical frame can receive different attention scores as its relative temporal distance to the current frames changes, and that masking different heads induces unequal generation degradation. Motivated by these findings, we propose Focused Forcing, a training-free KV selection method that focuses cached history along both generated-frame and head dimensions. For each generated frame, Focused Forcing preserves the most relevant and distinctive historical frames by combining attention scores with diversity scores of historical frames, while assigning larger budgets to heads with higher estimated importance. Across multiple autoregressive generation paradigms, Focused Forcing achieves up to 1.48x end-to-end acceleration without training, while improving visual quality and text alignment. Our code is available at https://github.com/caipeiliang2004/Focused-Forcing.
Sources
- MonarchRT: Efficient Attention for Real-Time Video Generation
- MAGI-1: Autoregressive Video Generation at Scale
- Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets
- LESA: Learnable Stage-Aware Predictors for Diffusion Model Acceleration
- SkyReels-V2: Infinite-length Film Generative Model
- Context Forcing: Consistent Autoregressive Video Generation with Long Context
- One Step Diffusion via Shortcut Models
- Mean Flows for One-step Generative Modeling
- Efficient Autoregressive Video Diffusion with Dummy Head
- LTX-Video: Realtime Video Latent Diffusion
- Denoising Diffusion Probabilistic Models
- DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference
- Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model
- Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference
- Controllable Video Generation: A Survey
- Follow-Your-Motion: Video Motion Transfer via Efficient Spatial-Temporal Decoupled Finetuning
- Scalable Diffusion Models with Transformers
- Movie Gen: A Cast of Media Foundation Models
- Seedance 2.0: Advancing Video Generation for World Complexity
- FORA: Fast-Forward Caching in Diffusion Transformer Acceleration
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs