Evaluating Budgeted Context Projection with Unexecuted Companion Runs
cs.AI
Submitted: 2026-09-25
Updated: 2026-10-08
Comments: 23 pages. Revised analysis and presentation; expanded sensitivity analyses and offline reproducibility materials. Project code: https://github.com/timwhitez/ReVer_Pi. Source archive includes ancillary data and offline reanalysis scripts
Code: https://github.com/earendil-works/pi
Project page: https://nvlabs.github.io/SoL-Pi
License: http://creativecommons.org/licenses/by/4.0/
The gist: Context projection can shorten individual requests while changing whether an agent finishes within its budget.
Terminology
Abstract
Context projection can shorten individual requests while changing whether an agent finishes within its budget. We examine how sequential evaluation obscures this trade-off when a capped first continuation prevents its companion from running. In a recorded ReVerPi source-reading campaign, 15 pairs with two final answers yield 12 historically scored successes per arm. Retaining all 27 intervention boundaries distinguishes observed failures from ten unexecuted companions and bounds projected-minus-full success between-9 and +1 tasks. Under the archived scoring contract, a frozen projection selector has a success difference from full context of [-3,0]; outside four fitting tasks, it is [-4,-1] across 23 boundaries. Excluding one task whose platform premise is not established by retained actor-input evidence changes the nonfitting range to [-3,0] across 22 boundaries, so strict inferiority is not robust to that exclusion. Among eleven historically joint-success pairs, projection uses 25% fewer aggregate logical tokens but more tokens for the median pair and 55 rather than 35 suffix requests. These retrospective results concern one adaptively assembled campaign, not population performance. The case motivates accounting that retains every boundary, preserves unknown outcomes and policy dependencies, checks task premises, and separates bounded completion from success-conditioned resource use.
Sources
- SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent Harness
- The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management
- Towards Reliable Context Compression for Long-Horizon Agents: An Empirical Study of Execution Instability
- MemGPT: Towards LLMs as Operating Systems
- Unlocking Context Constraints of LLMs: Enhancing Context Efficiency of LLMs with Self-Information-Based Content Filtering
- ACON: Optimizing Context Compression for Long-horizon LLM Agents
- AI Agents That Matter
- Token Reduction Is Not Cost Reduction
- Deep Reinforcement Learning at the Edge of the Statistical Precipice
- Distribution-Free, Risk-Controlling Prediction Sets
- Time-uniform, nonparametric, nonasymptotic confidence sequences
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems