Agent-Facing Information Design in LLM Tool Registries: A Preregistered Test of Rhetoric, Position and Structure
cs.IR, cs.AI, econ.GN, q-fin.EC
Submitted: 2026-04-12
Updated: 2026-09-30
Comments: 16 pages, 4 figures, 7 tables. v2 extend the v1 results with a preregistered confirmatory study on two OpenAI models
License: http://creativecommons.org/licenses/by/4.0/
The gist: AI agents often pick tools from registries, where each tool's provider writes its description.
Terminology
Abstract
AI agents often pick tools from registries, where each tool's provider writes its description. We ask whether sales language in those descriptions changes which tool an agent picks. We built pairs of listings differing in one controlled way (added praise, a verifiable specification, or list order) and asked two OpenAI models to call one tool. In a preregistered study, stacked praise (four kinds combined) raised a tool's pick rate by about 43 percentage points, matching or beating a verifiable specification. Praise also pulled some picks toward tools that could not do the task, but rarely toward tools asking for unneeded data access. With identical listings, the first-listed tool was picked about 72 points more often. On tasks with numeric limits, structured fields helped agents pick the capable tool; adding the provider's sales text beside the fields reduced or erased that gain. Registries could list limits as fields, hide sales text from agents, and randomize order. Stacked praise, but no single kind, replicated on held-out domains. Results are provisional until blind phrase ratings are complete, and cover two small models.
Sources
- What Is Your AI Agent Buying? Evaluation, Biases, Model Dependence, & Emerging Implications for Agentic E-Commerce
- A Framework for Studying AI Agent Behavior: Evidence from Consumer Choice Experiments
- Model Context Protocol (MCP) Tool Descriptions Are Smelly! Towards Improving AI Agent Efficiency with Augmented MCP Tool Descriptions
- Dynamics of Adversarial Attacks on Large Language Model-Based Search Engines
- Manipulating Large Language Models to Increase Product Visibility
- Adversarial Search Engine Optimization for Large Language Models
- Prompt Injection Attack to Tool Selection in LLM Agents
- ToolTweak: An Attack on Tool Selection in LLM-based Agents
- Whom Do AI Agents Work For? Role Assignment Induces Sponsorship Bias in LLM Recommenders
- MCPGuard : Automatically Detecting Vulnerabilities in MCP Servers
Related papers
- The Price of Isolation: Estimating the Ecosystem Cost of Symmetric Two-Sided A/B Testing
- SCAR: Semantic Continuity-Aware Retrieval for Efficient Context Expansion in RAG
- MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corpora
- How Do LLM-Generated Texts Impact Term-Based Retrieval Models?
- MemSyco-Bench: Benchmarking Sycophancy in Agent Memory
- Training with Missed Targets in Generative Recommendation: Separating Supervision from Probability Competition