A Baseline Assessment for the Agentic Web Technology Stack
cs.AI, cs.ET, cs.SE
Submitted: 2026-10-09
Updated: 2026-10-09
Code: https://github.com/kaushikb11/awesome-llm-agents2https:
Project page: https://webmachinelearning.github.io/webmcp
License: http://creativecommons.org/licenses/by/4.0/
The gist: The fast pace in which protocols, frameworks, and technologies are being proposed for LLM-based agents results in a complex scenario for anyone aiming to create state-of-the-art AI solutions.
Abstract
The fast pace in which protocols, frameworks, and technologies are being proposed for LLM-based agents results in a complex scenario for anyone aiming to create state-of-the-art AI solutions. Without an assessment on how different protocols, frameworks, and libraries work together to compose an Agentic Web solution, developers and organizations face different challenges (e.g., skilling, governance, infrastructure) even before becoming part of the Agentic Web ecosystem. In this context, we propose a system for performing Agentic Web evaluations called MASTIF (Multi-Agent System TestIng Framework) and present a baseline assessment of Agentic Web technologies, including 3 LLMs, 3 protocols, and 4 agent implementation frameworks, against 300 Mind2Web benchmark tasks. Results suggest that selecting a task-specific technology stack may result in gains of task completion rate, efficiency score, and social value stability scores; and reductions in verbosity and time to complete tasks. With the presented results, we expect to present a baseline in terms of metrics and results for this complex and fast evolving scenario for those aiming at designing, assessing, and developing multi-agent systems.
Sources
- Assessing and Verifying Task Utility in LLM-Powered Applications
- Benchmarking LLM powered Chatbots: Methods and Metrics
- Clembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agents
- Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
- Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark
- DeepSeek-V3 Technical Report
- PentestGPT: An LLM-empowered Automatic Penetration Testing Tool
- Mind2Web: Towards a Generalist Agent for the Web
- A survey of agent interoperability protocols: Model Context Protocol (MCP), Agent Communication Protocol (ACP), Agent-to-Agent Protocol (A2A), and Agent Network Protocol (ANP)
- Sampling Algorithms, from Survey Sampling to Monte Carlo Methods: Tutorial and Literature Review
- AutoAgent: A Fully-Automated and Zero-Code Framework for LLM Agents
- Mistral 7B
- AI Agents That Matter
- ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents
- Preference Leakage: A Contamination Problem in LLM-as-a-judge
- Tachikuma: Understading Complex Interactions with Multi-Character and Novel Objects by Large Language Models
- AgentSims: An Open-Source Sandbox for Large Language Model Evaluation
- AgentBench: Evaluating LLMs as Agents
- RoCo: Dialectic Multi-Robot Collaboration with Large Language Models
- Improving Grounded Language Understanding in a Collaborative Environment by Interacting with Agents Through Help Feedback
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems