MEMENTO: Leveraging Web as a Learning Signal for Low-Data Domains
cs.AI
Submitted: 2026-05-28
Updated: 2026-10-05
Project page: https://behavior-in-the-wild.github.io/SDR-Bench.html
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Real-world tasks often lack large labeled datasets, motivating extensive work on learning in low-data regimes.
Terminology
Abstract
Real-world tasks often lack large labeled datasets, motivating extensive work on learning in low-data regimes. Existing approaches such as few-shot prompting, instruction tuning, and synthetic data generation, continue to treat labeled or pseudo-labeled data as the primary learning signal. In contrast, human practitioners acquire expertise through repeated, self-directed interaction with the open web, progressively refining both domain knowledge and search strategies. We propose MEMENTO, a framework that treats the web as a learning signal rather than a stateless retrieval interface. MEMENTO operates at two levels: within each session, it conducts iterative web exploration via an Adaptive Exploration Tree (AET) that decomposes tasks into evolving questions and reflects on intermediate findings; across sessions, it accumulates experience through dual-channel memory, separating declarative knowledge (facts) from procedural knowledge (search strategies). This design enables agents to learn reusable research strategies and domain expertise from trajectories of web interaction without additional model training. We evaluate MEMENTO on three structurally distinct low-data domains: Sales Automation, Legal Outcome Prediction, and Subpopulation Opinion Prediction. Our empirical results show consistent improvement in performance over ReAct baselines (+25.6% on sales, +36.5% on legal research, and 29.6% TVD reduction on opinion prediction), demonstrating that the web can serve as a scalable learning source for acquiring task-specific expertise in data-scarce settings.
Sources
- GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
- JUSTICE: A Benchmark Dataset for Supreme Court's Judgment Prediction
- ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning
- Textbooks Are All You Need
- Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
- SE-Search: Self-Evolving Search Agent via Memory and Dense Reward
- WebCoach: Self-Evolving Web Agents with Cross-Session Memory Guidance
- WebGPT: Browser-assisted question-answering with human feedback
- R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning
- Finetuned Language Models Are Zero-Shot Learners
- Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning
- Qwen3 Technical Report
- ReAct: Synergizing Reasoning and Acting in Language Models
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems