Hybrid Latent Attention for Looped Language Models
cs.CL, cs.AI
Submitted: 2026-10-06
Updated: 2026-10-06
Sources
- MathArena: Evaluating LLMs on Uncontaminated Math Competitions
- Lessons from the Trenches on Reproducible Evaluation of Language Models
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- Let's Verify Step by Step
- TransMLA: Multi-Head Latent Attention Is All You Need
- Looped Latent Attention: Cross-Loop KV Compression for Looped Transformers
- Fast Transformer Decoding: One Write-Head is All You Need
- RoFormer: Enhanced Transformer with Rotary Position Embedding
- FlashLoop: Fast and Memory-Efficient Looped Transformers via Lazy Updates
- Scaling Latent Reasoning via Looped Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering