SkillCommit: Evolving Agent Skills through Behaviorally Validated Scope Expansion

The gist The proposed SkillCommit framework is an online skill evolution framework that continuously transforms experience into a hierarchical library of reusable skills, ensuring that new abstractions preserve validated behavior through rigorous replay evidence.

How it works

SkillCommit organizes skill evolution into three distinct stages: Feedback-Guided Patch Induction, Behavioral Compatibility Grouping, and Source-Preserving Consolidation The first stage, Feedback-Guided Patch Induction, derives an instance-specific patch based on a failed trajectory and locally validates it by replaying this task instance This process involves two steps: Patch Generation and Source-replay Validation In Patch Generation, a teacher model produces a reference trajectory that diagnoses the discrepancy between the failed and successful behaviors of the executor, leading to a candidate patch containing concise procedural guidance Each candidate patch ci is validated by replaying the source instance with the same frozen executor; if it successfully solves xi, it is admitted as a source-validated instancespecific patch This generateand-replay cycle repeats for at most K = 3 attempts, with high success rates for yielding a validated patch

Behavioral Compatibility Grouping

This stage determines how a locally-validated patch can be reused beyond its original instance by evaluating whether it is behaviorally compatible with an existing higher-level skill If a compatible skill is found, the framework attempts to expand that skill to cover the new patch Otherwise, it searches for a group of compatible residual patches and uses them to induce a new higher-level skill Semantic similarity is used only to retrieve a bounded set of plausible candidates, but the actual decision is based on behavioral replay and mechanism assessment because semantically similar patches may implement incompatible strategies For expanding an existing skill, the compatibility score C(p, qt) is computed as ρ(p, xt) + Pj∈W(p)ρ(qt, xj)/1 + W(p)

Source-Preserving Consolidation

This final stage abstracts behaviorally compatible patches into a higher-level skill and commits it only when it preserves the validated behavior of every constituent skill A candidate is committed over its proposed scope only if it preserves all previously validated behavior within that scope, otherwise, its scope may be narrowed to a replay-supported subset or the candidate is rejected The transitions for consolidation include Commit, Split, and Reject If a candidate successfully replays on every source instance within its proposed scope, it is committed as a higher-level skill If replay succeeds on only a subset of the proposed sources, SkillCommit selects the candidate version with the strongest replay support and may split it if it contains at least m instances If no candidate yields a replay-supported subset of at least m source instances, or if the narrowed candidate fails revalidation, it is rejected

Experiments and Results

SkillCommit was evaluated across RuleArena, OpenExempt, and KOR-Bench to assess its performance on diverse reasoning domains SkillCommit consistently improves every evaluated setting over no-skill execution and other baseline methods Table 1 summarizes evaluation results showing that SkillCommit raises the unweighted mean of the reported task metrics from 50.80% to 78.15% across all evaluated subsets Further experiments show that the induced skills transfer effectively across model scales and retain utility even when transferred across model families In-family cross-scale transfer demonstrates that learned skills preserve a reusable procedural core across model scales, while part of their effectiveness still depends on executor-specific interpretation and prompting conventions Cross-family and cross-scale transfer shows that GPT-5.4 authored skills raise the Qwen3.5-9B average from 48.97 without skills to 69.75, demonstrating transfer across both model family and scale boundaries The component ablation analysis confirms that Source-replayed patch induction supplies the dominant performance gain, while Source-Preserving Consolidation acts as a preservation gate Overall, the results indicate that behavioral induction and consolidation provide a portable interface between heterogeneous models without completely eliminating familyand executor-specific conventions

Conclusion

SkillCommit presents an online skill evolution framework that transforms accumulated agent experience into a compact, hierarchical skill library without sacrificing previously validated behavior Rather than relying solely on semantic similarity or model judgment, SkillCommit grounds consolidation in behavioral evidence, admitting every broader behavioral claim only after it survives replay Experiments on RuleArena, OpenExempt, and KOR-Bench demonstrate consistent improvements over no-skill settings and strong skill-evolution baselines The resulting skills also transfer across model scales and families, suggesting that procedural knowledge can be authored by more capable models and reused by smaller, more economical agents This represents a practical step toward agents that keep improving from experience while their accumulated knowledge remains compact, reusable, and behaviorally reliable.

--- Page 1 ---

SkillCommit: Evolving Agent Skills through Behaviorally Validated Scope Expansion

Expansion

Yu He1

, Weikai Yang2∗

1Nanjing University

2The Hong Kong University of Science and Technology (Guangzhou)

yu he@smail.nju.edu.cn, weikaiyang@hkust-gz.edu.cn

Abstract

Large language model (LLM) agents can continually improve without parameter updates by converting historical experience into reusable procedural knowledge However, existing methods often consolidate experience based on semantic similarity or LLM judgments, which may merge superficially related but behaviorally incompatible strategies and thereby degrade performance To address the issue, we propose SkillCommit, an online skill evolution framework that continuously transforms experience into a hierarchical library of reusable skills Each new experience is initially preserved as an instancespecific patch, retaining the behavior validated in its local context As related skills accumulate, SkillCommit abstracts those sharing a common behavioral mechanism into higherlevel skills Specifically, for each incoming skill, embeddingbased retrieval first identifies candidate related skills Crossinstance replay and an LLM-based mechanism check determine whether these skills transfer across cases and share a common underlying mechanism Candidates that pass both checks are abstracted into a higher-level skill and committed only if it preserves the validated behavior of all constituent skills Experiments on RuleArena, OpenExempt and KOR-Bench demonstrate that SkillCommit consistently improves agent performance across diverse domains Moreover, the learned skills transfer across model scales and families, enabling cross-model experience transfer

Introduction

Large language model (LLM) agents can use external tools and interact with their environments with increasing autonomy (Yao et al. 2023; Schick et al. 2023; Yang et al. 2023; Deng et al. 2023; Zhou et al. 2024) Yet reliable deployment in specialized domains often requires domain knowledge, procedural conventions, and operational constraints that base models do not acquire during pretraining Reusable skill provide a modular way to externalize such procedures and guide agents without modifying model parameters (Wang et al. 2025; Ni et al. 2026; Ma et al. 2026; Shinn et al. 2023; Zhao et al. 2024; Ouyang et al. 1> In practice, however, such knowledge is rarely available as a complete instruction document and instead remains scattered across practitioners’ experience, execution histories, and task feedback Manually eliciting and formalizing such knowledge is costly and may∗ still omit rare but critical boundary cases This gap motivates skill evolution, in which agents automatically extract, consolidate, and refine reusable skills from accumulated trajectories and feedback (Ouyang et al. 2026; Zhang et al. 2026a,b; Chen et al. 2026; Suzgun et al. 2026) Existing methods typically use LLM-based abstraction to distill individual trajectories and feedback into instancespecific patches that agents can retrieve and reuse when solving similar tasks (Shinn et al. 2023; Zhao et al. 2024; Ouyang et al. 1) However, since these patches are narrow in scope, retaining them individually leads to growing memory and retrieval costs as experience accumulates To build a more compact and general skill library, subsequent methods consolidate related instance-specific patches into higher-level skills based on semantic similarity (Xu et al. 2025; Ma et al. 2026) or model judgments (Xu et al. 2025; Wang et al. 1; Ni et al. 2026; Ma et al. 1; Chen et al.

1. No longer consolidate experience based solely on semantic similarity or LLM judgments, which may merge superficially related but behaviorally incompatible strategies and thereby degrade performance. SkillCommit ensures that abstractions are only committed if they preserve the validated behavior of all constituent skills.

2. Implement a three-stage online framework: Feedback-Guided Patch Induction, Behavioral Compatibility Grouping, and Source-Preserving Consolidation to continuously transform experience into a hierarchical library of reusable skills.

3. Introduce Feedback-Guided Patch Induction where a teacher model generates a candidate patch based on the discrepancy between the agent's failed trajectory and the ground truth, followed by validation via Source-replay Validation to ensure it successfully solves the instance.

4. Employ Behavioral Compatibility Grouping using cross-instance replay and mechanism assessment to determine if patches share a common underlying mechanism, rather than relying on surface similarity alone for skill abstraction.

5. Ensure that when abstracting, the framework only commits a higher-level skill if it preserves the validated behavior of every constituent skill, preventing the loss of critical context during generalization.

6. Enable cross-model experience transfer by ensuring learned skills transfer across model scales and families, allowing smaller executors to utilize expertise authored by more capable models, as demonstrated in experiments where GPT-5.4-authored skills improved Qwen3.5-9B performance on certain tasks.