AutoBuddy: Evaluating Proactive, Always-On Agents on Task Discovery and Execution in a Simulated Workplace
cs.AI, cs.LG
Submitted: 2026-11-09
Updated: 2026-11-09
Code: https://github.com/github/copilot-sdk
Terminology
Sources
- $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment
- UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
- ARE: Scaling Up Agent Environments and Evaluations
- CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environments
- CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions
- ProEvent: An Event-centric Benchmark for Proactive Agents
- MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers
- Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
- Proactive Agent Research Environment: Simulating Active Users to Evaluate Proactive Assistants
- Beyond Reactivity: Measuring Proactive Problem Solving in LLM Agents
- GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks
- AutomationBench
- ProAgentBench: Evaluating LLM Agents for Proactive Assistance with Real-World Data
- OdysseyBench: Evaluating LLM Agents on Long-Horizon Complex Office Application Workflows
- BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents
- Ask Now, Use Later: Benchmarking the Proactivity Gap in Long-Lived LLM Agents
- TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks
- $\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents