MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents
cs.CL, cs.AI, cs.LG
Submitted: 2026-02-02
Updated: 2026-11-06
Comments: Accepted by NeurIPS 2026. Code is available at https://github.com/ViktorAxelsen/MemSkill
Code: https://github.com/ViktorAxelsen/MemSkill
License: http://creativecommons.org/licenses/by/4.0/
The gist: Most Large Language Model (LLM) agent memory systems rely on a small set of static, hand-designed operations for extracting memory.
Terminology
Abstract
Most Large Language Model (LLM) agent memory systems rely on a small set of static, hand-designed operations for extracting memory. These fixed procedures hard-code human priors about what to store and how to revise memory, making them rigid under diverse interaction patterns and inefficient on long histories. To this end, we present MemSkill, which reframes these operations as learnable and evolvable memory skills, structured and reusable routines for extracting, consolidating, and pruning information from interaction traces. Inspired by the design philosophy of agent skills, MemSkill employs a controller that learns to select a small set of relevant skills, paired with an LLM-based executor that produces skill-guided memories. Beyond learning skill selection, MemSkill introduces a designer that periodically reviews hard cases where selected skills yield incorrect or incomplete memories, and evolves the skill set by proposing refinements and new skills. Together, MemSkill forms a closed-loop procedure that improves both the skill-selection policy and the skill set itself. Experiments on LoCoMo, LongMemEval, HotpotQA, and ALFWorld demonstrate that MemSkill improves task performance over strong baselines and generalizes well across settings. Further analyses shed light on how skills evolve, offering insights toward more adaptive, self-evolving memory management for LLM agents. Code is available at https://github.com/ViktorAxelsen/MemSkill
Sources
- Multi-Agent Evolve: LLM Self-Improve through Co-evolution
- Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
- LightMem: Lightweight and Efficient Memory-Augmented Generation
- The Llama 3 Herd of Models
- Automated Design of Agentic Systems
- Memory in the Age of AI Agents
- R-Zero: Self-Evolving Reasoning LLM from Zero Data
- Memory OS of AI Agent
- A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts
- Evaluating Very Long-Term Conversational Memory of LLM Agents
- AlphaEvolve: A coding agent for scientific and algorithmic discovery
- ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory
- Proximal Policy Optimization Algorithms
- ALFWorld: Aligning Text and Embodied Environments for Interactive Learning
- Mem-{\alpha}: Learning Memory Construction via Reinforcement Learning
- RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning
- Agent Workflow Memory
- Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory
- LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
- EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecycle
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data
- BALTO: Balanced Token-Level Policy Optimization for Hallucination Mitigation
- Hidden Measurement Error in LLM Pipelines Distorts Annotation, Evaluation, and Benchmarking