How Can Recommendation Feedback Evolve Agent Memory?
cs.AI
Submitted: 2026-09-29
Updated: 2026-09-30
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Content-generation agents continuously receive impressions, clicks, conversions, and negative feedback from recommendation systems, providing real-world outcome signals for memory evolution.
Terminology
Abstract
Content-generation agents continuously receive impressions, clicks, conversions, and negative feedback from recommendation systems, providing real-world outcome signals for memory evolution. However, these signals are delayed and noisy, confounded by audience composition, placement, and recommendation policies, and may result from the combined influence of multiple memories, making accurate attribution difficult. Existing methods rely primarily on immediate feedback or semantic retrieval and therefore struggle to reliably translate recommendation outcomes into memory fitness. To address this challenge, we propose TIDE (Trajectory-Informed Directed Memory Evolution), an external memory evolution framework driven by delayed recommendation feedback. We further introduce Memory Evolution Gain (MEG), which measures the utility improvement of evolved memory over a no memory baseline on strictly future tasks. TIDE treats memory as a capacity-constrained population of experiences: temporal and semantic credit assignment estimates contextual fitness, while responsibility credit distributes outcome signals according to the memories referenced during generation. These signals are then used to reinforce, crossover, mutate, or evict memories. On an e-commerce membership marketing content-generation agent, TIDE achieves a +7.75-percentage-point MEG in offline temporal replay and significantly improves both unique click-through rate (UCTR) and activation rate in an online A/B test. On a delayed-label benchmark, TIDE achieves the lowest mean absolute error (MAE) and root mean squared error (RMSE) and the highest MEG among the compared methods, demonstrating its effectiveness.
Sources
- PosterOmni: Generalized Artistic Poster Creation via Task Distillation and Unified Reward Feedback
- CTR-Driven Advertising Image Generation with Multimodal Large Language Models
- CTR-Driven Ad Text Generation via Online Feedback Preference Optimization
- PosterReward: Unlocking Accurate Evaluation for High-Quality Graphic Design Generation
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Learning Personalized Agents from Human Feedback
- Let's Verify Step by Step
- Training language models to follow instructions with human feedback
- ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Reflexion: Language Agents with Verbal Reinforcement Learning
- Kimi k1.5: Scaling Reinforcement Learning with LLMs
- EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective
- Agent Workflow Memory
- Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory
- A-MEM: Agentic Memory for LLM Agents
- MemoryCD: Benchmarking Long-Context User Memory of LLM Agents for Lifelong Cross-Domain Personalization
- Absolute Zero: Reinforced Self-play Reasoning with Zero Data
- Self-Adapting Language Models
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems