SLATE: A Large-Scale Benchmark for Long-Horizon API Selection in E-commerce Tool Use
cs.CL, cs.AI
Submitted: 2026-11-02
Updated: 2026-11-02
Code: https://github.com/strands-labs/strands-decider
Project page: https://geshijoker.github.io/slate
Terminology
Sources
- FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Models
- MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
- Emergent autonomous scientific research capabilities of large language models
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
- $\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
- ALFWorld: Aligning Text and Embodied Environments for Interactive Learning
- API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs
- ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases
- Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios
- MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use
- TRAJECT-Bench:A Trajectory-Aware Benchmark for Evaluating Agentic Tool Use
- The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution
- ACEBench: Who Wins the Match Point in Tool Usage?
- ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities
- SOP-Bench: Complex Industrial SOPs for Evaluating LLM Agents
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data