HERALD: High-Throughput Block Diffusion LLM Inference via CPU-GPU Cooperative KV Cache Retrieval
cs.LG
Submitted: 2026-06-19
Updated: 2026-11-01
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- LLaDA2.0: Scaling Up Diffusion Language Models to 100B
- SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation
- Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions
- CDLM: Consistency Diffusion Language Models For Faster Sampling
- MAGE: All-[MASK] Block Already Knows Where to Look in Block Diffusion LLM
- Fast Inference from Transformers via Speculative Decoding
- LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference
- dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching
- Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction
- Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding
- Dream 7B: Diffusion Large Language Models
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis