Accelerating LLM Decoding with In-DRAM Parallelism
cs.AR
Submitted: 2026-11-02
Updated: 2026-11-02
Terminology
Sources
- SkipDecode: Autoregressive Skip Decoding with Batching and Caching for Efficient LLM Inference
- Gemma 4 Technical Report
- The Llama 3 Herd of Models
- Mistral 7B
- Mixtral of Experts
- MVDRAM: Enabling GeMV Execution in Unmodified DRAM for Low-Bit LLM Acceleration
- Term Revealing: Furthering Quantization at Run Time on Quantized DNNs
- The Remarkable Robustness of LLMs: Stages of Inference?
- KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache
- Qwen3.5-Omni Technical Report
- Fast Transformer Decoding: One Write-Head is All You Need
- GLU Variants Improve Transformer
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated Parameters
- Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference
- CQIL: Inference Latency Optimization with Concurrent Computation of Quasi-Independent Layers
Related papers
- Embedded Arena: Iterative Optimization via Hardware Feedback
- WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantization
- SparsePixels: Efficient Convolution for Sparse Data on FPGAs
- Golden Ruler: A Numeric Format Catalog with Bit-Exact Conformance Vectors for FP8, BF16, MXFP4, and Microscaling Formats
- AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving
- RFChipAgent: Multi-Agentic AI Flow for Analog/RF Chip Design