R squared Flow: Recursive Self-Improvement via Recursive Skill Evolution

arXiv:2609.33867 · cs.AI · Submitted 2026-09-27 · Read on arXiv

cs.AI

Submitted: 2026-09-27

Updated: 2026-09-27

Comments: 27 pages

Code: https://github.com/beita6969/r2flow

License: http://creativecommons.org/licenses/by/4.0/

The gist: LLM-based agents can improve themselves across tasks by reusing and revising the skills they orchestrate into executable procedures.

Terminology

Abstract

LLM-based agents can improve themselves across tasks by reusing and revising the skills they orchestrate into executable procedures. Flow-based training fits this loop: it samples procedures in proportion to reward, and the flow through each skill credits it for the next library revision. Three obstacles stand in the way of making this self-improvement reliable: flow training suffers strategy collapse over tree-structured histories; nonnegative flow-based credit rewards frequent use as if it were benefit; and library edits rest on the task reward the policy optimizes. We introduce R squared Flow, a recursive self-improvement framework that alternates policy learning, independent verification, and versioned skill-library updates on a shared-state orchestration graph. The graph merges histories that differ only in the order of independent steps, allowing flow training to pool evidence across equivalent executions. A flow-share readout of the trained flow, invariant to the backward policy, and a separate signed utility rank which skills to change, verifier evidence decides whether an edit is warranted, and a residual-variance plateau sets when to update. Committed edits reshape the graph the next policy learns on, realizing recursive skill evolution. Across question answering, mathematical reasoning, interactive decision making, and code generation, R squared Flow improves task accuracy and library-edit precision over heuristic orchestration, reinforcement learning, and skill-evolution baselines, and transfers across executors. Code is available at https://github.com/beita6969/r2flow.

Sources

Related papers