ExperienceIndex: Artifact-Grounded Memory
cs.CL, cs.AI, cs.IR, cs.LG
Submitted: 2026-10-07
Updated: 2026-10-07
License: http://creativecommons.org/licenses/by/4.0/
The gist: Knowledge-intensive tasks require answering many questions by reasoning about a shared corpus of artifacts (e.g., court cases, or scientific literature).
Abstract
Knowledge-intensive tasks require answering many questions by reasoning about a shared corpus of artifacts (e.g., court cases, or scientific literature). As humans interact with these corpora, they naturally accumulate experiential knowledge about artifacts, enabling them to quickly identify the complete set of relevant artifacts for each new task. However, existing AI agents lack appropriate memory solutions to build or reuse such artifact-grounded experience, leading to lower answer quality and higher online cost. Existing memory solutions extract and reuse information from prior task-solving traces, but they primarily focus on user preferences, factual attributes, or abstract reasoning patterns rather than persistent artifact-specific knowledge. We introduce ExperienceIndex, a novel experience layer for AI agents that captures and reuses knowledge about artifacts based on prior reasoning traces. ExperienceIndex stores two complementary forms of experience: (i) single-artifact experiences that summarize an artifact's contribution to prior tasks and (ii) artifact-pair experiences that encode structural relationships discovered during past reasoning. Integrated as lightweight middleware, ExperienceIndex uses an experience retrieval mechanism to guide agents toward the complete set of relevant artifacts for new tasks, improving both answer quality and efficiency. Across diverse corpora and agentic solutions with different search frameworks, ExperienceIndex delivers consistent gains, raising answer quality by up to 11.0 points and reducing online dollar cost by up to 50.5%. We further demonstrate two benefits: (i) cross-task generalization, where experiences accumulated from text-to-SQL tasks transfer to factoid QA tasks over the same artifact corpus, and (ii) teacher-student learning, where experiences from a stronger model enable a weaker model to reach comparable performance.
Sources
- Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
- AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite
- KARL: Knowledge Agents via Reinforcement Learning
- EnrichIndex: Using LLMs to Enrich Retrieval Indices Offline
- Log-Augmented Generation: Scaling Test-Time Reasoning with Reusable Computation
- Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
- From Local to Global: A Graph RAG Approach to Query-Focused Summarization
- OpenAI o1 System Card
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- PAR$^2$-RAG: Planned Active Retrieval and Reasoning for Multi-Hop Question Answering
- ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory
- REMem: Reasoning with Episodic Memory in Language Agent
- Full-Pipeline Inference Optimization for MiMo-V2.5 Series: Pushing Hybrid SWA Efficiency to the Limit
- FreshStack: Building Realistic Benchmarks for Evaluating Retrieval on Technical Documents
- Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering