RidgeFill: Training-Free Late-Layer Cache Fill for Faster Prefill
cs.LG, cs.CL
Submitted: 2026-11-06
Updated: 2026-11-06
Code: https://github.com/ApproximateScience/llama.cpp-ridgefill
Terminology
Sources
- LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
- LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks
- Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
- LexGLUE: A Benchmark Dataset for Legal Language Understanding in English
- Accelerating Large Language Model Decoding with Speculative Sampling
- A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents
- DeepSeek-V3 Technical Report
- DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
- Enhancing Chat Language Models by Scaling High-quality Instructional Conversations
- LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference
- The Llama 3 Herd of Models
- POP: Prefill-Only Pruning for Efficient Large Model Inference
- Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse
- KV Prediction for Improved Time to First Token
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention
- FastKV: Decoupling of Context Reduction and KV Cache Compression for Prefill-Decoding Acceleration
- Efficient Memory Management for Large Language Model Serving with PagedAttention
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis