RoMeRL: Balancing Feedback Coverage and the Memory-Reward Trap in Self-Evolving Agent Memory via Reduced-Order Utility States
Yi Yang, Zhennan Chen, Yihong Zhuang, Tiehan Fan, Yinan Chen, Jian Li, Jian Yang, Ying Tai
cs.LG, cs.CL
Submitted: 2026-08-10
Updated: 2026-08-11
Code: https://github.com/YOUNG-fnxm/RoMeRL
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- FLEX: Continuous Agent Evolution via Forward Learning from Experience
- Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
- A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents
- Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Management
- ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory
- ALFWorld: Aligning Text and Embodied Environments for Interactive Learning
- Cognitive Architectures for Language Agents
- Voyager: An Open-Ended Embodied Agent with Large Language Models
- Agent Workflow Memory
- AtomMem: Building Simple and Effective Memory System for LLM Agents via Atomic Facts
- Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents
- MemEvolve: Meta-Evolution of Agent Memory Systems
- MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory
- Learn to Memorize: Optimizing LLM-based Agents with Adaptive Memory Framework
- LifelongAgentBench: Evaluating LLM Agents as Lifelong Learners
- Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering
- Memento: Fine-tuning LLM Agents without Fine-tuning LLMs
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks