DC-SAE: Deep Compression Semantic Autoencoder for Faster Diffusion Convergence
cs.CV
Submitted: 2026-09-30
Updated: 2026-09-30
Comments: 16 pages, 5 figures. Project page: https://dagroup-pku.github.io/DCSAE
Code: https://github.com/DAGroup-PKU/DCSAE
Project page: https://dagroup-pku.github.io/DCSAE
License: http://creativecommons.org/licenses/by/4.0/
The gist: High-compression tokenizers are essential for scaling latent image generative models.
Terminology
Abstract
High-compression tokenizers are essential for scaling latent image generative models. However, aggressive compression creates a fundamental tradeoff between reconstruction fidelity and generation efficiency: high compression image encoder always increases the learning difficulty of diffusion training, resulting in slow model convergence. Recent representation autoencoders speed up the diffusion training by improving the latent feature's expressive capability by replacing VAE encoders with pretrained semantic encoders, yet they are typically limited to moderate compression and lose pixel-level details necessary for faithful reconstruction. To achieve both high compression and fast diffusion training, we propose DC-SAE, a Decoupled Compact Semantic Autoencoder designed for high-compression image generation with accelerated diffusion model convergence. DC-SAE consists of two key components: (1) a macro-level architecture design that leverages semantic encoders to enable higher compression ratios, and (2) a pixel-level encoder that preserves low-level details, ensuring high-fidelity image reconstruction. We empirically demonstrate that DC-SAE performs strongly on image generation tasks, achieving both compact latent representations and efficient training dynamics. Specifically, on the ImageNet dataset with 512 times 512 resolution, DC-SAE achieves 32 times spatial compression, with 29.79 PSNR and 3.37 gFID, substantially outperforming the previous state-of-the-art high-compression tokenizer baselines DC-AE by 13.5% and 54.9% on PSNR and gFID, respectively, maintaining comparable throughput and faster diffusion model training convergence. Beyond class-conditional generation, a 1.6 B-parameter DiT using DC-SAE achieves 0.84 on GenEval and 86.007 on DPG-Bench for text-to-image generation at 1024 times1024 resolution.
Sources
- FlexTok: Resampling Images into 1D Token Sequences of Flexible Length
- Qwen3-VL Technical Report
- DC-AE 1.5: Accelerating Diffusion Model Convergence with Structured Latent Space
- Emu3.5: Native Multimodal Models are World Learners
- Unified Autoregressive Visual Generation and Understanding with Continuous Tokens
- Seed1.5-VL Technical Report
- ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment
- Ming-UniVision: Joint Image Understanding and Generation with a Unified Continuous Tokenizer
- LatentUM: Unleashing the Potential of Interleaved Cross-Modal Reasoning via a Latent-Space Unified Model
- Auto-Encoding Variational Bayes
- TC-AE: Unlocking Token Capacity for Deep Compression Autoencoders
- Autoregressive Image Generation without Vector Quantization
- DGAE: Diffusion-Guided Autoencoder for Efficient Latent Representation Learning
- Delving into Latent Spectral Biasing of Video VAEs for Superior Diffusability
- UniTok: A Unified Tokenizer for Visual Generation and Understanding
- LongCat-Next: Lexicalizing Modalities as Discrete Tokens
- Cosmos World Foundation Model Platform for Physical AI
- Latent Diffusion Model without Variational Autoencoder
- DINOv3
- UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editing
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models