Mosaic: Unlocking Over 30 times Context Length for Diffusion LLMs Inference via Global Memory Planning and Dynamic Peak Taming
cs.LG
Submitted: 2026-01-10
Updated: 2026-11-08
Terminology
Sources
- The Llama 3 Herd of Models
- DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
- UltraLLaDA: Scaling the Context Length to 128K for Diffusion Large Language Models
- FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion
- A Survey on Large Language Model Acceleration based on KV Cache Management
- LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs
- dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching
- dInfer: An Efficient Inference Framework for Diffusion Language Models
- Large Language Models: A Survey
- Large Language Diffusion Models
- Efficient Memory Management for Deep Neural Net Inference
- Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding
- Dream-Coder 7B: An Open Diffusion Language Model for Code
- Qwen3 Technical Report
- Dream 7B: Diffusion Large Language Models
- PAT: Accelerating LLM Decoding via Prefix-Aware Attention with Resource Efficient Multi-Tile Kernel
- Discrete Diffusion in Large Language and Multimodal Models: A Survey
- A Survey of Large Language Models
- LLaDA-MoE: A Sparse MoE Diffusion Language Model
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis