MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGs

arXiv:2605.08374 · cs.AI · Submitted 2026-05-08 · Read on arXiv

cs.AI

Submitted: 2026-05-08

Updated: 2026-10-12

Comments: 32 pages, 13 figures, including appendices

Code: https://github.com/jwliao-ai/MemQ

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Provenance — 3× in this paper · explained in 5 episodes
• Benchmark — 1× in this paper · explained in 2 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Multimodal Reasoning — 1× in this paper · explained in 5 episodes
• Q-learning — 1× in this paper · explained in 3 episodes
• Trace — 1× in this paper · explained in 2 episodes

Sources

• Model-Free Episodic Control
• Remember Me, Refine Me: A Dynamic Procedural Memory Framework for Experience-Driven Agent Evolution
• Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
• Memp: Exploring Agent Procedural Memory
• RecallM: An Adaptable Memory Mechanism with Temporal Understanding for Large Language Models
• Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Management
• What Deserves Memory: Adaptive Memory Distillation for LLM Agents
• MemGPT: Towards LLMs as Operating Systems
• Gemini Robotics: Bringing AI into the Physical World
• Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory
• Mem-T: Densifying Rewards for Long-Horizon Memory Agents
• MemEvolve: Meta-Evolution of Agent Memory Systems
• MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents
• MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory
• Live-Evo: Online Evolution of Agentic Memory from Continuous Feedback
• Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Tasks
• LifelongAgentBench: Evaluating LLM Agents as Lifelong Learners
• Memento: Fine-tuning LLM Agents without Fine-tuning LLMs
• MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• Agents' Last Exam
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems