Weights That Stay: On-Chip Megakernels for Serial Heads in Speculative Decoding
cs.DC, cs.LG
Submitted: 2026-10-31
Updated: 2026-10-31
Code: https://github.com/tatsu-lab/stanford_alpaca
Terminology
Sources
- Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding
- Accelerating Large Language Model Decoding with Speculative Sampling
- Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
- From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline
- Let's Verify Step by Step
- XPress: Parallel Refinement for Diffusion Drafters in Speculative Decoding
- FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving
- DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Drafting
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
- SGLang: Efficient Execution of Structured Language Model Programs
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends