DRAMA: Dissecting Attention Redundancy for Accelerating Multimodal Diffusion Large Language Models
cs.LG, cs.AI
Submitted: 2026-11-12
Updated: 2026-11-12
Code: https://github.com/CindyHung20/DRAMA
Terminology
Sources
- Structuring The Future: Diffusion LLM Speculative Decoding via Calibrated Draft Graphs
- Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
- Hallucination of Multimodal Large Language Models: A Survey
- Enabling Approximate Joint Sampling in Diffusion LMs
- DPad: Efficient Diffusion Language Models with Suffix Dropout
- dParallel: Learnable Parallel Decoding for dLLMs
- Saber: An Efficient Sampling with Adaptive Acceleration and Backtracking Enhanced Remasking for Diffusion Language Model
- MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
- Learnable Sampler Distillation for Discrete Diffusion Models
- Self Speculative Decoding for Diffusion Large Language Models
- Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMs
- FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion
- Empirical Analysis of Decoding Biases in Masked Diffusion Models
- Planned Diffusion
- d$^2$Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
- Accelerating Diffusion LLM Inference via Local Determinism Propagation
- Mercury: Ultra-Fast Language Models Based on Diffusion
- FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference
- SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension
- Diffusion Language Models Know the Answer Before Decoding
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis