RSIGym: A Flexible Environment for Recursive Self-Improvement

arXiv:2610.10310 · cs.LG · Submitted 2026-10-07 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "RSIGym: A Flexible Environment for Recursive Self-Improvement".

Jane: The gist: RSIGym introduces an agent-native research environment based on Everything as a Service (EaaS) that allows agents to investigate individual interventions and jointly optimize data, training settings,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So we're looking at this paper now called RSIGym: A Flexible Environment for Recursive Self-Improvement. I mean, it sounds like they’re building something designed specifically to let agents keep getting better by themselves without needing a massive setup every single time.

Jane: It really does sound like that, Tom. The title suggests it's about a flexible environment that lets agents improve recursively. It’s not just one big training run, but a whole system for iterative self-improvement where the agent proposes changes and then actually implements those changes in a controlled way.

Lu: What I see immediately is this focus on agent-native research infrastructure. They're tackling the problem that existing setups often leave agents to rebuild routine infrastructure or restrict their exploration to just one tiny component at a time.

Meng: Exactly. If you want an AI to really iterate and improve itself, it needs a way to manage those changes without breaking everything every time it tries something new.

Lalam: It sounds like they're giving the agent the tools not just to train, but also to test its own training process in a structured loop.

Tom: Right. So what’s the core idea of this RSIGym setup? What exactly is this environment doing that makes it different from just running a standard training script?

The paper's summary: Jane: Well, the paper explains that RSIGym is built around an agent-native research loop. Think of it as a continuous cycle where the agent proposes a change, trains or revises its candidate, evaluates that candidate with its current harness, and then uses the results from that evaluation to decide what to try next in the next step.

Tom: That’s key. It’s not just one-shot training anymore; it’s an iterative process driven by the agent's own feedback. And they call this loop a research loop on page three of THIS PAPER > RSIGym: A Flexible Environment for Recursive Self-Improvement.

Lu: They expose five reusable services through Everything as a Service, or EaaS. These services include things like a Train Server that runs LoRA fine-tuning through Tinker, and then a Model Server that exposes base models and trained checkpoints via an OpenAI compatible gateway.

Meng: So the agent can just call these services on demand without having to manage the low-level details of how those components are actually running on hardware or software. That sounds like it simplifies the engineering burden significantly for us in production.

Lalam: And there’s also a Rollout Server for frontier models, which is used for data generation and judging, plus a Sandbox Service offering E2B cloud sandboxes.

Tom: And all of these services are connected by a shared authorization service that checks every single call against the run’s permissions and remaining budget > RSIGym: A Flexible Environment for Recursive Self-Improvement page three of that work reads: Authorization one key per run ⋅ permission check before each call · shared budget billed after each call verify / bill service calls.

Jane: So, in plain terms, they’ve created a single place where the agent can investigate individual interventions—like changing just the data or just the training settings—or it can jointly optimize data, training settings, and execution harnesses all at once.

The paper's improvements: Tom: The main thing they propose is these different tracks of improvement. They call them the Data track, where the agent changes the training data but keeps the recipe and harness fixed.

Lu: Then there’s the Harness track, where it revises execution code with model weights held fixed, and then there’s a Joint track that opens up data, training settings, and harness code together. That’s how they study co-evolution.

Meng: I like the separation of tracks because it lets you isolate variables. If the performance drops in the joint track, you can figure out if it was the data interaction or the harness revision causing it.

Jane: And to measure success in these different spaces, they introduce something called RSI-Index across five benchmarks: software engineering, terminal interaction, mathematics, scientific reasoning and skill-based tasks > RSIGym: A Flexible Environment for Recursive Self-Improvement page one of that work reads: We define RSI-Index as the mean fraction of the remaining performance gap closed across five benc <ref:2610.10310#pg1,We define RSI-Index as the mean fraction of the remaining performance>.

Lalam: They show a concrete result in the Joint track where Opus five achieves an RSI-Index of zero point four eight zero nine under a platform service budget of a five hundred per benchmark run > RSIGym: A Flexible Environment for Recursive Self-Improvement page nine of that work reads: In the Joint track at five hundred per run: all six research agents improve Qwen3 point 5-35B-A3B-Base, and no agent is best everywhere.

Tom: That Opus five result is interesting because it shows that its selected systems actually improve all five benchmarks, raising SWE-bench Verified from seventeen point six seven percent to fifty point three three percent and AIME from thirty-one point six seven percent to ninety-seven point seven eight percent. Those are big jumps in performance metrics > RSIGym: A Flexible Environment for Recursive Self-Improvement page nine of that work reads: This result shows that its selected systems improve all five benchmarks, raising SWE-bench Verified from seventeen point six seven percent to fifty point three three percent and AIME from thirty-one point six seven percent to ninety-seven point seven eight percent.

Jane: So the environment isn't just theoretical; they’re showing empirical evidence that this system can drive real, measurable performance gains across multiple domains simultaneously when used in the Joint track > RSIGym: A Flexible Environment for Recursive Self-Improvement page nine of that work reads: This result shows that its selected systems improve all five benchmarks, raising SWE-bench Verified from seventeen point six seven percent to fifty point three three percent and AIME from thirty-one point six seven percent to ninety-seven point seven eight percent.

Conclusion: Tom: So, to wrap up the RSIGym paper, it’s a system that combines reusable services with budget and permission controls to support both component-level interventions and joint optimization of data, training settings, and harnesses > RSIGym: A Flexible Environment for Recursive Self-Improvement page two of that work reads: Training data and hyperparameters determine the learned model, and the harness determines how that model reasons, uses tools, and acts in a task <ref:2610.10310#pg1,RSIGym: A Flexible Environment for Recursive Self-Improvement>.

Jane: It’s designed to support both successful improvements and unsuccessful research attempts because it lets agents diagnose failures through recorded trajectories > RSIGym: A Flexible Environment for Recursive Self-Improvement page three of that work reads: Inside a lightweight container, the research agent iterates a research loop: it generates data, trains a checkpoint, evaluates the checkpoint with its harness, and analyzes the results to revise its data, training configuration, or harness.

Lu: The study shows that outcomes vary depending on things like the starting harness or how much budget you have available > RSIGym: A Flexible Environment for Recursive Self-Improvement page two of that work reads: Reducing this engineering burden must go hand in hand with preserving the agent’s freedom to explore how different components contribute to improvement <ref:2610.10310#pg1,RSIGym: A Flexible Environment for Recursive Self-Improvement>.

Meng: And they even show that heavy training doesn't always mean better performance; in eight out of ten comparisons, the more heavily trained candidate scores lower > RSIGym: A Flexible Environment for Recursive Self-Improvement page fourteen of that work reads: Heavier training does not consistently improve performance, as the more heavily trained candidate scores lower in eight of ten comparisons.

Lalam: It’s clear that the environment supports both successful improvements and the study of unsuccessful research attempts > RSIGym: A Flexible Environment for Recursive Self-Improvement page one of that work reads: We introduce RSIGym, an agent-native research environment based on Everything as a Service (EaaS) <ref:2610.10310#pg1,We introduce RSIGym, an agent-native research environment based on Everything as>.

Tom: It’s a lot of infrastructure work, but it seems they’ve managed to make the agent feel autonomous enough to handle the complexity > RSIGym: A Flexible Environment for Recursive Self-Improvement page one of that work reads: RSIGym: A Flexible Environment for Recursive Self-Improvement Fanqing Meng1,2,∗, Lingxiao Du1,2,∗, Haocheng Lu1, Qiguang Chen1 Ziqi Zhao1, Zijian Wu2, Jiayuan Zhuo1, Mengkang Hu1,† Michael Qizhe Shieh1,2,† <ref:2610.10310#pg1,RSIGym: A Flexible Environment for Recursive Self-Improvement Fanqing Meng1,2,∗ , Lingxiao>

Jane: It’s a lot of infrastructure work to give an AI the freedom to explore its own path without needing human intervention > RSIGym: A Flexible Environment for Recursive Self-Improvement page three of that work reads: Inside a lightweight container, the research agent iterates a research loop: it generates data, trains a checkpoint, evaluates the checkpoint with its harness, and analyzes the results to revise its data, training configuration, or harness.

Fanqing Meng, Lingxiao Du, Haocheng Lu, Qiguang Chen, Ziqi Zhao, Zijian Wu, Jiayuan Zhuo, †Mengkang Hu, ›Michael Qizhe Shieh

Evolvent AI National University of Singapore

cs.LG

Submitted: 2026-10-07

Updated: 2026-10-07

Code: https://github.com/BerriAI/litellm

Project page: https://lee1003-lee.github.io/Awesome-RSI-Research/assets/rsi-survey.pdf

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 88/100

The gist: The gist: RSIGym introduces an agent-native research environment based on Everything as a Service (EaaS) that allows agents to investigate individual interventions and jointly optimize data, training

Key concepts

Agent-Native Research Loop
This is the core research cycle where an agent acts as the researcher. It involves the agent proposing a change, executing it (like training or code revision), evaluating the result against benchmarks, and then using that evidence to decide on the next experiment. This loop allows for continuous, autonomous self-improvement of systems.
Everything as a Service (EaaS)
EaaS is the underlying architecture where all necessary research services are provided as reusable components. These services include servers for training models, serving base models, running frontier models for data generation, and evaluating benchmarks. This modular setup allows agents to interact with various tools seamlessly.
Improvement Tracks
The environment supports different ways to guide research through distinct tracks: Data track (changing data only), Harness track (revising execution code only), and Joint track (optimizing data, training settings, and harness together). Agents can operate within these tracks or jointly across them using shared budget controls.
RSI-Index
The RSI-Index is a metric used to measure agent achievement across various benchmarks. It calculates the average fraction of the remaining performance gap closed on each benchmark. A higher RSI-Index indicates that the system being researched is improving more effectively across all tested areas.

Summary

The gist: RSIGym introduces an agent-native research environment based on Everything as a Service (EaaS) that allows agents to investigate individual interventions and jointly optimize data, training settings, and execution harnesses within the same environment

RSIGym Architecture

RSIGym is designed around an agent-native research loop where the research agent works in a lightweight, CPU-only container and conducts experiments through five reusable services The core components include a Train Server that executes LoRA fine-tuning through Tinker, a Model Server exposing base models and trained checkpoints via an OpenAI-compatible gateway, a Rollout Server providing frontier models for data generation and judging, a Benchmark Server evaluating model–harness pairs through Harbor, and a Sandbox Service offering E2B cloud sandboxes These services support the continuous research loop where the agent proposes a change, trains or revises a candidate, evaluates it, and uses the returned evidence to decide what to try next

Research Tracks and Control

The environment supports different tracks of improvement through shared budget and permission controls The available tracks include the Data track, where the agent changes training data while the training recipe and harness remain fixed, the Harness track, where it revises execution code with model weights held fixed, and a Joint track that opens data, training settings, and harness code together The shared authorization service checks every call against the run’s permissions and remaining budget and records its cost This design enables agents to investigate individual interventions and jointly optimize data, training settings, and execution harnesses within the same environment

Evaluation Metrics

To measure agent achievements within these improvement spaces, the paper introduces RSI-Index across five benchmarks covering software engineering, terminal interaction, mathematics, scientific reasoning, and skill-based tasks The index averages the fraction of the remaining performance gap closed on each benchmark In the Joint track, Opus 5 achieves the highest RSI-Index of 0.4809 under a platform-service budget of a 500 per benchmark run This result shows that its selected systems improve all five benchmarks, raising SWE-bench Verified from 17.67% to 50.33% and AIME from 31.67% to 97.78%

Experimental Findings

Experiments reveal how outcomes vary with the starting harness, available budget, and access to external data With Qwen3.6-35B-A3B-Instruct weights fixed, autonomous revisions to the existing DSH-harness raise Terminal-Bench performance from 30.34% to 40.82%, increasing the number of tasks solved on all three attempts from 15 to 26 The recorded trajectories connect these outcomes to the agents’ experiments, showing how they diagnose failures, compare candidates, and select final systems

Key Contributions

The paper's contributions are threefold First is an Agent-Native RSI Environment that combines reusable research services with shared budget and permission controls to support both component-level interventions and joint optimization Second is Cross-Domain Evaluation and Empirical Analysis, which defines RSI-Index, compares six frontier research models across five benchmarks, and analyzes how research decisions and experimental settings shape improvement Third is Open Code and Experimental Records, releasing the full codebase to support reproducibility and further research

Case Study Insights

The case studies demonstrate that harness improvements address recurring execution failures, such as those in SWE or Terminal-Bench by checking edits before patch submission or preserving shell state Heavier training does not consistently improve performance, as the more heavily trained candidate scores lower in eight of ten comparisons Research behavior differs across model families, with Claude agents spending more time diagnosing failures and revising the harness DeepSeek shows another effective route by combining low-cost data generation with diagnosis of training and tool-interface failures The environment supports both successful improvements and the study of unsuccessful research attempts

Conclusion

RSIGym introduces an agent-native environment that combines reusable research services with shared budget and permission controls It supports individual interventions and joint optimization of data, training settings, and harnesses The results show that the environment supports both successful improvements and the study of unsuccessful research attempts

--- Page 7 ---

RSIGym

compute budget for investigating the target system.

--- Page 8 ---

RSIGym

Table 3: Runs in this study. Each run is an independent search that submits one system. Budget is the per-run allocation for platform services. The first four rows are reported in this section; the last three are the case studies of Section 5

--- Page 9 ---

RSIGym

Figure 2: Joint track at 500 per run: all six research agents improve Qwen3.5-35B-A3B-Base, and no agent is best everywhere. Each cell gives the official score and, as a bar and a percentage, the share of the gap to a perfect score that is closed

--- Page 10 ---

RSIGym

Figure 3: Single tracks and a doubled budget. Each row joins two official scores: the initial and the submitted system, or the 500 and the 1000 run of the same agent

--- Page 12 ---

RSIGym

Figure 4: Training comparisons within Joint runs. Each comparison uses the same harness and task set: a trained candidate against base weights, or a heavier against a lighter update

--- Page 13 ---

RSIGym

Figure 5: Research time and normalized gain of every 500 Joint run. Each bar is one run’s session, from the agent’s first to its last step; the official evaluation is excluded. The number is the run’s normalized gain, in blue for the best agent on the benchmark and red below the initial system

--- Page 14 ---

RSIGym

Figure 6: Three case studies. Blue is above the reference shown in grey, red below; stars mark the submitted system

--- Page 15 ---

RSIGym

(a) Data run without network access checkpoints: development, 1 attempt 0.00 0.05 0.10 0.15 official baseline 0.1049

--- Page 16 ---

RSIGym

(b) DSH harness research with Qwen3.6-35B-A3B-Instruct fixed; bands join matching categories, not individual tasks

--- Page 17 ---

RSIGym

(c) Qwen3.8-27B improving itself SkillsBench reward 0.3 0.4 0.5 0.6 0.7

--- Page 18 ---

RSIGym

(a) Data run without network checkpoints: development, official baseline

--- Page 19 ---

RSIGym

(b) DSH harness research with Qwen3.6-35B-A3B-Instruct fixed; bands join matching categories, not individual tasks

--- Page 20 ---

RSIGym

(c) Qwen3.8-27B improving itself SkillsBench reward

--- Page 21 ---

RSIGym

(a) Data run without network checkpoints: development, official baseline

--- Page 22 ---

RSIGym

(b) DSH harness research with Qwen3.6-35B-A3B-Instruct fixed; bands join matching categories, not individual tasks

--- Page 23 ---

RSIGym

(c) Qwen3.8-27B improving itself SkillsBench reward

--- Page 24 ---

RSIGym

(a) Data run without network checkpoints: development, official baseline

--- Page 25 ---

RSIGym

(b) DSH harness research with Qwen3.6-35B-A3B-Instruct fixed; bands join matching categories, not individual tasks

--- Page 26 ---

RSIGym

(c) Qwen3.

Improvements for AI systems

  1. A recursive self-improvement loop is enabled by RSIGym, an agent-native research environment that combines reusable research services with shared budget and permission controls. This allows agents to investigate individual interventions and jointly optimize data, training settings, and execution harnesses within the same environment.

  2. Agents can execute a continuous research loop where they propose a change, trains or revises a candidate, evaluates it, and uses the returned evidence to decide what to try next, managed through services like the Training Service (LoRA fine-tuning via Tinker) and Evaluation Service (Harbor).

  3. The environment supports controlled studies of different research tracks: the agent can operate in the Data track where they change training data while the training recipe and harness remain fixed, or in the Harness track where they revise the execution code with model weights held fixed.

  4. Performance gains are measured by RSI-Index, which is defined as the mean fraction of the remaining performance gap closed across five benchmarks covering software engineering, terminal interaction, mathematics, scientific reasoning, and skill-based tasks.

  5. The system can dynamically adjust its research focus based on budget constraints; for instance, the paper shows that varying total budget while holding permitted changes and evaluation protocols fixed shows how the results of a research agent scale with the resources available to it.

  6. Agents can diagnose failures through recorded trajectories, as recorded trajectories reveal how agents diagnose failures and select candidates, allowing them to learn from unsuccessful research attempts.

Abstract

Recursive self-improvement requires carrying accepted changes into later improvement cycles, while studying agent-proposed changes also requires substantial research infrastructure. Existing settings often leave agents to rebuild routine infrastructure or restrict exploration to individual components. We introduce RSIGym, an agent-native research environment based on Everything as a Service (EaaS). RSIGym exposes training, inference, rollout, evaluation, and sandbox execution through reusable services, with shared budget and permission controls supporting Data, Harness, and Joint improvement tracks. This design enables agents to investigate individual interventions and jointly optimize data, training settings, and execution harnesses within the same environment. We define RSI-Index as the mean fraction of the remaining performance gap closed across five benchmarks covering software engineering, terminal interaction, mathematics, scientific reasoning, and skill-based tasks. Comparing six frontier research models in independent Joint runs, Opus 5 achieves the highest RSI-Index of 0.4809 under a 500 platform-service budget per benchmark run. Its selected systems improve all five benchmarks, raising SWE-bench Verified from 17.67% to 50.33% and AIME from 31.67% to 97.78%. Additional experiments examine DSH-harness refinement, budget variation, and restricted network access, while recorded trajectories reveal how agents diagnose failures and select candidates. We open-source the full RSIGym codebase and results to support reproducibility and further research.

Sources

Related papers