SKILL-KD: Contrastive Skill Distillation for LLM Agents

arXiv:2607.28048 · cs.AI · Submitted 2026-07-30 · Read on arXiv

cs.AI

Submitted: 2026-07-30

Updated: 2026-10-13

Terminology

• Distillation — 3× in this paper · explained in 2 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Benchmark — 1× in this paper · explained in 2 episodes
• Large Language Model (LLM) — 1× in this paper · explained in 3 episodes
• Trace — 1× in this paper · explained in 2 episodes

Sources

• EvoSkill: Automated Skill Discovery for Multi-Agent Systems
• SkillRet: A Large-Scale Benchmark for Skill Retrieval in LLM Agents
• SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine
• SkillReducer: Optimizing LLM Agent Skills for Token Efficiency
• LiveMathematicianBench: A Live Benchmark for Research-Level Mathematical Reasoning with Proof Sketches
• Graph-of-Skills: Dependency-Aware Structural Retrieval for Massive Agent Skills
• SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolution
• SkillGen: Verified Inference-Time Agent Skill Synthesis
• Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills
• AgentDistill: Training-Free Agent Distillation with Generalizable MCP Boxes
• SkillX: Automatically Constructing Skill Knowledge Bases for Agents
• SkillGrad: Optimizing Agent Skills Like Gradient Descent
• Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents
• WebXSkill: Skill Learning for Autonomous Web Agents
• SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning
• Self-Distilled RLVR
• OPID: On-Policy Skill Distillation for Agentic Reinforcement Learning
• SkillOpt: Executive Strategy for Self-Evolving Agent Skills
• TextGrad: Automatic "Differentiation" via Text
• SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• Agents' Last Exam
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems