Balancing Memory Pathways: Analyzing and Improving Memory Utilization in Hybrid LMs
cs.CL, cs.AI
Submitted: 2026-10-05
Updated: 2026-10-05
Comments: Code: https://github.com/amy-hyunji/Balancing-Memory-Pathways
Code: https://github.com/amy-hyunji/Balancing-Memory-Pathways
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Sources
- Just read twice: closing the recall gap for recurrent language models
- Retrieval-Aware Distillation for Transformer-SSM Hybrids
- Component Ablation for Efficient Hybrid Language Model Architectures: Performance, Resilience, and Compression Implications
- BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning
- TextWorld: A Learning Environment for Text-based Games
- Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
- A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers
- Unified Language Model Pre-training for Natural Language Understanding and Generation
- Hymba: A Hybrid-head Architecture for Small Language Models
- DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
- In-context Autoencoder for Context Compression in a Large Language Model
- CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
- LoRA: Low-Rank Adaptation of Large Language Models
- Kimi Linear: An Expressive, Efficient Attention Architecture
- The NarrativeQA Reading Comprehension Challenge
- Understanding and Enhancing Mamba-Transformer Hybrids for Memory Recall and Language Modeling
- MINTEval: Evaluating Memory under Multi-Target Interference in Long-Horizon Agent Systems
- 500xCompressor: Generalized Prompt Compression for Large Language Models
- Some Attention is All You Need for Retrieval
- Learning to Compress Prompts with Gist Tokens
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering