Dissecting model behavior through agent trajectories
cs.AI, cs.LG
Submitted: 2026-06-16
Updated: 2026-10-01
Code: https://github.com/strands-labs/benchmark-harnesses
Terminology
Sources
- A General Path-Based Representation for Predicting Program Properties
- Language Models are Few-Shot Learners
- DeepSeek-V3 Technical Report
- SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
- GraphCodeBERT: Pre-training Code Representations with Data Flow
- Qwen2.5-Coder Technical Report
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- CODESTRUCT: Code Agents over Structured Action Spaces
- Large Language Models are Zero-Shot Reasoners
- Efficient Memory Management for Large Language Model Serving with PagedAttention
- RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems
- AgentBench: Evaluating LLMs as Agents
- An Analysis of the Search Spaces for Generate and Validate Patch Generation Systems
- Data Contamination: From Memorization to Exploitation
- Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
- SWE-PolyBench: A multi-language benchmark for repository level evaluation of coding agents
- NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark
- Toolformer: Language Models Can Teach Themselves to Use Tools
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems