From Lookups to Lessons: Turning Retrieved Knowledge into Parametric Memory
cs.CL, cs.AI, cs.LG
Submitted: 2026-10-30
Updated: 2026-10-30
Code: https://github.com/SSRand/From-Lookups-to-Lessons
Terminology
Sources
- Cartridges: Lightweight and general-purpose long context representations via self-study
- Distilling the Knowledge in a Neural Network
- Mistral 7B
- TransMem: Transforming Hidden States into Memory for Large Language Models
- delta-mem: Efficient Online Memory for Large Language Models
- Tuning Language Models by Proxy
- TS-Memory: Plug-and-Play Memory for Time Series Foundation Models
- MemGPT: Towards LLMs as Operating Systems
- Learning by Distilling Context
- MemSFT: Mitigating Alignment Tax with an External Parametric Memory
- Learning What to Remember: Test-Time Training via Context Distillation
- MLP Memory: A Retriever-Pretrained Memory for Large Language Models
- Memory Decoder at Scale: A Pretrained, Parametric Long-Term Memory
- OPSDL: On-Policy Self-Distillation for Long-Context Language Models
- Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data