InvestigationWorlds: An Agentic Environment for Legal Investigation
cs.AI, cs.CL, cs.CY, cs.LG
Submitted: 2026-10-02
Updated: 2026-10-02
Comments: Accepted to NeurIPS 2026 Evaluations & Datasets
Code: https://github.com/harveyai/harvey-labs
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: We introduce InvestigationWorlds, an agentic environment for legal investigation.
Abstract
We introduce InvestigationWorlds, an agentic environment for legal investigation. We build on an underused artifact of U.S. civil litigation: the summary judgment motion. This motion relies upon a record composed of real evidence exhibits, and results in a court-adopted hypothesis that is treated as ground truth for the purposes of deciding the motion. Each environment is built from a real U.S. Federal Court case retrieved from Public Access to Court Electronic Records (PACER) and augmented by an attorney-validated generation pipeline that synthesizes role-tagged documents around the original record. The resulting corpus admits multiple coherent factual readings, only one of which matches the court-adopted hypothesis. Evaluating on 100 cases, we find agents often commit to incorrect hypotheses despite retrieving relevant evidence, struggling to distinguish the court-adopted hypothesis from alternative hypotheses.
Sources
- Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models
- LawBench: Benchmarking Legal Knowledge of Large Language Models
- TextArena
- LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
- CorporateBench: Large-Scale Q&A Benchmarking with Temporal Knowledge Bases
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- GEM: A Gym for Agentic LLMs
- Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools
- Better Call GPT, Comparing Large Language Models Against Lawyers
- GAIA: a benchmark for General AI Assistants
- LegalBench-RAG: A Benchmark for Retrieval-Augmented Generation in the Legal Domain
- KORGym: A Dynamic Game Platform for LLM Reasoning Evaluation
- MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoning
- REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewards
- MuSiQue: Multihop Questions via Single-hop Question Composition
- BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents
- CBR-RAG: Case-Based Reasoning for Retrieval Augmented Generation in LLMs for Legal Question Answering
- HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
- $\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
- Benchmarking Multi-Step Legal Reasoning and Analyzing Chain-of-Thought Effects in Large Language Models
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems