Towards Optimal Multi-draft Speculative Decoding
cs.CL, cs.DS
Submitted: 2025-02-26
Updated: 2026-11-02
Terminology
Sources
- Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
- Accelerating Large Language Model Decoding with Speculative Sampling
- Evaluating Large Language Models Trained on Code
- Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding
- Cascade Speculative Drafting for Even Faster LLM Inference
- REST: Retrieval-Based Speculative Decoding
- Recursive Speculative Decoding: Accelerating LLM Inference via Sampling Without Replacement
- EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty
- PaSS: Parallel Speculative Sampling
- Lossless Acceleration of Large Language Model via Adaptive N-gram Parallel Decoding
- Accelerating LLM Inference with Staged Speculative Decoding
- TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding
- Block Verification Accelerates Speculative Decoding
- LLaMA: Open and Efficient Foundation Language Models
- Qwen2 Technical Report
- Inference with Reference: Lossless Acceleration of Large Language Models
- Multi-Candidate Speculative Decoding
- OPT: Open Pre-trained Transformer Language Models
- DistillSpec: Improving Speculative Decoding via Knowledge Distillation
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data