MCRI: A Four-Dimensional Framework for Analyzing and Evaluating Agent Skills
cs.AI, cs.SE
Submitted: 2026-10-01
Updated: 2026-10-01
Comments: 24PAGES
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: As agents evolve from single-tool systems into modular, composite architectures, skills are becoming an important mechanism for capability development and distribution.
Terminology
Abstract
As agents evolve from single-tool systems into modular, composite architectures, skills are becoming an important mechanism for capability development and distribution. However, the academic community lacks a structured framework for systematically analyzing and evaluating skills. Drawing on information gain and behavioral constraint, we propose the four-dimensional MCRI Framework and operationalize it as MCRI-Eval, a large language model-based evaluation method. We evaluate MCRI-Eval using 63,812 public skills from the OpenClaw skill Hub, with 58,275 skill-conditioned model executions across BigCodeBench, BFCL-Fundamental, and Mind2Web. MCRI-Eval scores are positively associated with community popularity signals and achieve the highest downstream ranking agreement among the evaluated methods. MCRI-Eval also improves top-1 skill selection across all three benchmarks: compared with the strongest baseline on each benchmark, the skills selected by MCRI-Eval advance by 17.7, 22.8, and 19.6 percentile points in downstream performance rank on BigCodeBench, BFCL-Fundamental, and Mind2Web, respectively. These results indicate that MCRI-Eval provides a useful pre-execution signal for prioritizing promising skills before costly execution-based evaluation.
Sources
- Agent Skill Evaluation and Evolution: Frameworks and Benchmarks
- SkillVetBench: LLM-as-Judge for Multi-Dimensional Security Risk Evaluation in Open-Source LLM Agent Skills
- SkillSieve: A Hierarchical Triage Framework for Detecting Malicious AI Agent Skills
- SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks
- AgentBench: Evaluating LLMs as Agents
- How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings
- AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents
- Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills
- A Framework for Evaluating Agentic Skills at Scale
- Survey on Evaluation of LLM-based Agents
- OpenSkillEval: Automatically Auditing the Open Skill Ecosystem for LLM Agents
- PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems