Transect: Retaining Observability for Long-Horizon LLM Agent Evaluations
cs.AI
Submitted: 2026-10-06
Updated: 2026-10-08
Code: https://github.com/AI-Safety-Institute/transect
Project page: https://meridianlabs-ai.github.io/inspect_viz
Sources
- Open-World Evaluations for Measuring Frontier AI Capabilities
- Survey on Evaluation of LLM-based Agents
- Evaluation and Benchmarking of LLM Agents: A Survey
- Measuring AI Agents' Progress on Multi-Step Cyber Attack Scenarios
- How Inference Compute Shapes Frontier LLM Evaluation
- The Last Human-Written Paper: Agent-Native Research Artifacts
- HiBayES: A Hierarchical Bayesian Modeling Framework for AI Evaluation Statistics
- Stop Comparing LLM Agents Without Disclosing the Harness
- Harness-Bench: Evaluating Agent Harnesses across Models and Realistic Workflows
- Reliability without Validity: A Systematic, Large-Scale Evaluation of LLM-as-a-Judge Models Across Agreement, Consistency, and Bias
- Seven simple steps for log analysis in AI systems
- RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts
- AI Researchers' Views on Automating AI R&D and Intelligence Explosions
- Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops
- Automated alignment is harder than you think
- MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research
- Compound Deception in Elite Peer Review: A Failure Mode Taxonomy of 100 Fabricated Citations at NeurIPS 2025
- The Case of the Mysterious Citations
- Can AI agents conduct open-ended AI research? Early evidence from two case studies
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems