From Benchmarks to Behaviors: A Framework for Auditing AI Agents
cs.HC
Submitted: 2026-11-02
Updated: 2026-11-02
Code: https://github.com/earendil-works/pi
Terminology
Sources
- GEO: Generative Engine Optimization
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Large-Scale, Longitudinal Study of Large Language Models During the 2024 US Election Season
- Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
- AgentOps: Enabling Observability of LLM Agents
- Biased or Personalized? The Impact of Personal Information on AI-driven Development
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation
- Characterizing AI Agents for Alignment and Governance
- Manipulating Large Language Models to Increase Product Visibility
- Conversational AI increases political knowledge as effectively as self-directed internet search
- Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
- Agent Workflow Memory
- How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behavior
- ReAct: Synergizing Reasoning and Acting in Language Models
- $\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
- The Invisible Hand: Unveiling Provider Bias in Large Language Models for Code Generation
Related papers
- EmoPhone: A Multi-Wave Dataset for In-the-Wild Mobile and Wearable Affect Sensing
- Tasteply: Externalizing Stable and Situational Preference as Manipulable Spatial Objects
- Shielded Trajectory Sampling for Personalized Relational Safety in Conversational AI
- From Tasks to Turns: Measuring Human Labor Contributions in Human-Agent Joint Production
- Reconstruction and Reflection of Positive Experiences through Resurfacing Laughter-indexed Everyday Moments
- LLM-Assisted Reranking to Operationalize Nuanced Objectives in Recommender Systems