Looping Deeper Without Growing the KV Cache: Recurrent Depth for Causal Encoder-Decoders
cs.LG
Submitted: 2026-11-09
Updated: 2026-11-09
Terminology
Sources
- Longformer: The Long-Document Transformer
- CART: Context-Anchored Recurrent Transformer -- A Parameter-Efficient Architecture with Learned Stability
- Hybrid Latent Attention for Looped Language Models
- Memory-Efficient Looped Transformer: Decoupling Compute from Memory in Looped Language Models
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
- Towards Looped Models Done Right, Part I: Topology, Input Injection, Recurrent-State Design
- Towards Looped Models Done Right, Part II: Rethinking at Fixed Points
- LSTM-Style Looped Transformers with Shared KV Caches
- Asymmetric Looped Transformers: Fixed Global Representations for an Iterative Decoder
- Agentic Coding in the Wild: Characterizing GitHub Copilot Traces at Production Scale
- Muon is Scalable for LLM Training
- The Surprising Effectiveness of Shared Memory in Looped Transformers
- Looped Latent Attention: Cross-Loop KV Compression for Looped Transformers
- Parcae: Scaling Laws For Stable Looped Language Models
- BeNice! Closing Looping's Inference-Memory Gap via KV-Sharing
- Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving
- Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching
- Fast Transformer Decoding: One Write-Head is All You Need
- Universal YOCO for Efficient Depth Scaling
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis