Where KV-Cache Error Lives in Looped Language Models: A Diagnostic Study of Ouro and Huginn
cs.DC
Submitted: 2026-10-30
Updated: 2026-10-30
Comments: 11 pages of main text (37 pages including appendix), 5 figures, 37 tables
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
- Deep Equilibrium Models
- LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers
- Training Verifiers to Solve Math Word Problems
- Universal Transformers
- HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision
- LoopQ: Quantization for Looped Language Models
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
- Deep Learning with Limited Numerical Precision
- WaveFront Decoding: Parallelized Self-Speculative Decoding for Looped Language Models
- KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
- ResidualQuant: KV Cache Quantization for Looped Transformers with 2-Bit Residuals
- Efficient Memory Management for Large Language Model Serving with PagedAttention
- Shallow Queries, Mature Values: Depth-Asynchronous Self-Speculation for Looped Transformers
- Quantizing Looped Transformers: Feedback Exposure and Calibration Blindness
- Decoding Looped Transformers Better for (Almost) Free
- KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache
- Enabling Dynamic Computation in Looped LMs
- Up or Down? Adaptive Rounding for Post-Training Quantization
Related papers
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends
- AgentWeave: Efficient Distributed Agent Serving via Flow Decomposition