Towards Green Text Generation: Memory-Based Speculative Decoding
cs.CL
Submitted: 2026-11-12
Updated: 2026-11-12
Code: https://github.com/LanguageMachines/timbl
Terminology
Sources
- Accelerating Large Language Model Decoding with Speculative Sampling
- Mistral 7B
- Memory-based Language Models: An Efficient, Explainable, and Eco-friendly Approach to Large Language Modeling
- Inference with Reference: Lossless Acceleration of Large Language Models
- TinyLlama: An Open-Source Small Language Model
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data