SSD: Spatially Speculative Decoding Accelerates Autoregressive Image Generation
cs.CV
Submitted: 2026-06-18
Updated: 2026-09-27
Project page: https://shilongxiang.github.io/SSD/Abstract
License: http://creativecommons.org/licenses/by/4.0/
The gist: Autoregressive image models treat images as 1D token sequences, inheriting the next-token factorization of language models.
Terminology
Abstract
Autoregressive image models treat images as 1D token sequences, inheriting the next-token factorization of language models. This flattening discards a useful property of images: visual tokens are organized and correlated in two-dimensional space. We introduce Spatially Speculative Decoding (SSD), an inference-time decoding framework that exploits this spatial structure for accelerated generation. Rather than speculating only along the flattened sequence, SSD predicts both the adjacent horizontal token and the token directly below it, allowing multiple spatial directions to advance in parallel. This reduces the number of backbone forward evaluations and alleviates the memory bottleneck of autoregressive decoding. SSD accelerates image generation by up to 11.03x in wall-clock time while maintaining generation quality on DPG-Bench and GenEval. These results show that spatial structure provides a simple and effective source of parallelism for faster autoregressive image generation.
Sources
- Accelerating Large Language Model Decoding with Speculative Sampling
- Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling
- Distilling the Knowledge in a Neural Network
- Classifier-Free Diffusion Guidance
- ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment
- GPT-4o System Card
- Annealed Relaxation of Speculative Decoding for Faster Autoregressive Image Generation
- EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test
- DeepSeek-V3 Technical Report
- Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining
- L-MTP: Leap Multi-Token Prediction Beyond Adjacent Context for Large Language Models
- Multi-Scale Local Speculative Decoding for Image Generation
- Searching for Activation Functions
- GLU Variants Improve Transformer
- Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation
- Chameleon: Mixed-Modal Early-Fusion Foundation Models
- Emu3: Next-Token Prediction is All You Need
- Scaling Autoregressive Models for Content-Rich Text-to-Image Generation
- Locality-aware Parallel Decoding for Efficient Autoregressive Image Generation
- FlashAR: Efficient Post-Training Acceleration for Autoregressive Image Generation
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models