GRTO: Bootstrapped Group Relative Tool Optimization for Referring Segmentation

arXiv:2605.23500 · cs.CV, cs.LG · Submitted 2026-05-22 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "GRTO: Bootstrapped Group Relative Tool Optimization for Referring Segmentation".

Title and authors: Tom: So we're looking at this paper today called GRTO: Bootstrapped Group Relative Tool Optimization for Referring Segmentation. It sounds like they are tackling the tricky part of making models use tools, specifically segmentation tools, in a way that actually works well for complex language prompts.

Jane: That’s right. The name itself suggests they are using a bootstrapped approach to get that tool ready before the main policy training begins. It’s about how you make sure the model and the tool work together efficiently for those difficult referring segmentation tasks where the prompt is ambiguous or vague, right?

Lu: I think what's interesting here is how they frame it. They are not just looking at optimizing a single component; they are unifying reinforcement learning with differentiable auxiliary objectives for reasoning-intensive segmentation, which means the model and the tool update at the same time.

Meng: Unifying objectives sounds good on paper, but from my side, I wonder how much extra training signal is needed to make that joint optimization actually useful when you're dealing with large vision-language models.

Lalam: Based on what I'm seeing in the literature, this approach seems like it’s addressing the problem where you have a model that needs a tool but the tool itself isn't perfectly aligned yet, which is a common bottleneck in deploying these kinds of systems.

The paper's summary: Tom: Basically, the core idea of GRTO is to use group relative policy optimization rollouts to optimize an auxiliary tool objective while also optimizing the main policy. It uses this framework to let the decoder gradients complement the policy rewards, which sounds like a clever way to get better results than just training things separately.

Jane: I see what you mean. It’s about treating the segmentation tool not as something fixed, but as something that gets fine-tuned alongside the main reasoning model during training, so when the model learns better prompts, the tool learns to respond better too.

Lu: The summary points out that they use two stages: first bootstrapped tool optimization, which cheaply pre-aligns the tool to what they think is a post-trained policy, and then group relative tool optimization itself to fine-tune everything together. It’s a staged process for better convergence.

Meng: Pre-alignment sounds promising because it tackles the cost of training the policy and tool separately, but how do you define that "analytically optimal" bootstrapped objective when you can't actually compute the true reward function?

Lalam: They treat some parts of the reward function, like Z and the reward function R, as non-differentiable during the initial bootstrapping stage. Then, in the group relative tool optimization part, they use a stopgrad operation to reconstruct those gradients so they can integrate them into the main training loop.

The paper's improvements: Tom: The paper highlights that BTO allows for tool pretraining without being hindered by the cost of policy rollouts, which is a big win for efficiency. And then they show that B-GRTO, this two-stage bootstrapped method, leads to superior policy convergence and a more robust final system compared to plain GRPO.

Jane: So it’s not just about getting one piece better; it’s about the entire pipeline—the policy and the tool—getting better together in a way that plain methods can't match, especially in those challenging referring segmentation settings where things get messy.

Lu: They show that B-GRTO results in substantial improvements over plain GRPO, matching or even surpassing domain-specific state-of-the-art methods across three different referring segmentation settings. That’s a concrete comparison point for their performance claims.

Meng: If it actually beats those specific state-of-the-art methods on things like camouflage detection and remote sensing, that suggests the alignment mechanism is working effectively under real constraints, which is what I care about when we talk about deployment readiness.

Lalam: They achieve top two rankings across all metrics in camouflage detection on COD10K and NC4K, which gives a strong signal that this joint optimization strategy provides tangible performance gains over existing methods.

Conclusion: Tom: So, to wrap up the GRTO: Bootstrapped Group Relative Tool Optimization for Referring Segmentation paper, it shows that aligning the tool with the policy objective yields stronger, faster optimization, and they achieved state-of-the-art performance in camouflage detection and remote sensing segmentation.

Jane: It’s a strong result because they aren't treating the segmentation decoder as just some passive implementation detail; fine-tuning that tool actually gives them substantial performance gains even when the alignment isn't perfect, which is important for real world use.

Lu: They position B-GRTO as a general strategy for combining large language model reasoning with adaptable differentiable tools, and they acknowledge that their limitation is the risk of overfitting the decoder when you have small training sets in reasoning segmentation.

Meng: From an engineering standpoint, they are proposing a general strategy for using MLLM reasoning with adaptable tools, but I'm still concerned about how to implement that adaptive bootstrapping procedure in a way that doesn't require massive amounts of new data every time the policy improves.

Lalam: They successfully align a reasoning policy with a differentiable tool by showing that aligning the tool with the policy objective yields stronger, faster, and more robust optimization, establishing state-of-the-art performance where it matters most for referring segmentation.

Mario Markov, Stefan Maria Ailuro, Mohammad Mahdi, Luc Van Gool, Danda Pani Paudel

INSAIT, Sofia University

cs.CV, cs.LG

Submitted: 2026-05-22

Updated: 2026-11-18

The gist: Segmentation is a fundamental task in computer vision, underpinning pixel-level scene understanding and serving as a cornerstone for applications ranging from autonomous perception to medical image…

Key concepts

Group Relative Tool Optimization (GRTO)
A framework that jointly optimizes a policy with a differentiable tool. It reuses previous policy rollouts to optimize the auxiliary tool objective, allowing decoder gradients to help refine the main reasoning policy rewards during training.
Bootstrapped Tool Optimization (BTO)
A pretraining stage that cheaply aligns the segmentation tool before joint optimization. This allows the tool to be pre-aligned with what a theoretically post-trained policy should look like, speeding up convergence by handling initial tool alignment independently.
Referring Segmentation
A computer vision task where an AI must identify specific objects or regions in an image based on a text description. This requires both reasoning about the text and precise pixel-level segmentation to produce accurate masks.
Differentiable Tool Use
The process of integrating a tool (like a segmentation model) into the reinforcement learning policy such that the tool's output can be used to calculate gradients for updating both the policy and the tool itself. This allows both components to learn together effectively.

Terminology

Summary

Segmentation is a fundamental task in computer vision, underpinning pixel-level scene understanding and serving as a cornerstone for applications ranging from autonomous perception to medical image analysis <ref:2605. p. 023500#pg2>. B-GRTO introduces Bootstrapped-GRTO, a pretraining method that cheaply bootstraps the tool, leading to faster convergence and superior performance across three challenging referring segmentation settings <ref:2605. p. 023500#pg2>.

Framework Introduction

The paper introduces group relative tool optimization (GRTO), a mathematically grounded framework for jointly optimizing a policy with differentiable tool use <ref:2605.23500#pg2>. This framework reuses group relative policy optimization (GRPO) rollouts to optimize the auxiliary tool objective, allowing decoder gradients to complement policy rewards <ref:2605. p. 023500#pg2>. Further, Bootstrapped-GRTO (B-GRTO) is derived as a pretraining method that cheaply bootstraps the tool, resulting in faster convergence and superior performance compared to plain GRPO <ref:2605. p. 023500#pg2>.

Training Signal Construction

The optimization process involves two stages: bootstrapped tool optimization (BTO), which cheaply pre-aligns the tool to the theoretical post-trained policy, and group relative tool optimization (GRTO), which integrates tool fine-tuning into policy RL <ref:2605.23500#pg2>. The BTO objective is formulated as JBTO(ω) = −1/G X G i=1 1/Z e 1 β RiL(ψω), where Z and the reward function R are treated as non-differentiable <ref:2605. p. 423500#pg6>. The GRTO objective is defined as JGRTO(θ, ω) = JGRPO(θ, ωold) − 1/G X G i=1 Y oi t=1 r'i,tL(ψω), where r'i,t = stopgrad(ri,t) and the reward function R are treated as non-differentiable in order to reconstruct the gradients <ref:2605.23500#pg5>.

Performance Gains

B-GRTO results in substantial improvements over plain GRPO, matching or surpassing domain-specific state-ofPreprint methods across three challenging referring segmentation settings <ref:2605.23500#pg2>. Across camouflage detection, B-GRTO reaches state-of-the-art performance on COD10K and NC4K, ranking top-two across all metrics <ref:2605. p. 623500#pg8>. In remote sensing segmentation, GRTO and B-GRTO once again significantly enhance plain-GRPO <ref:2605. p. 623500#pg9>.

Methodology Details

The experiments utilize InternVL3.5-8B as the VLM backbone and SAM3 as the trainable segmentation tool <ref:2605. p. 423500#pg5>. The reward signal for the VLM is final-mask IoU, while the tool is trained using a differentiable BCE + soft-IoU surrogate loss <ref:2605. p. 423500#pg19>. The training involves setting KL-divergence weight β = 0. p. 501 and training until validation performance peaks <ref:2605. p. 523500#pg18>.

Discussion and Limitations

The paper concludes that the segmentation decoder should not be treated as a passive implementation detail, as fine-tuning SAM3 yields substantial performance gains even in misaligned regimes <ref:2605.23500#pg10>. B-GRTO is positioned as a general strategy for combining MLLM reasoning with adaptable differentiable tools <ref:2605. p. 823500#pg18>. Limitations include the risk of overfitting the decoder, particularly in reasoning segmentation where training set size is small <ref:2605. p. 823500#pg10>. Future work suggests more adaptive bootstrapping procedures, such as dynamic replay buffers refreshed as the policy improves <ref:2605. p. 823500#pg18>.

Conclusion

B-GRTO successfully aligns a reasoning policy with a differentiable tool, demonstrating that aligning the tool with the policy objective yields stronger, faster, and more robust optimization <ref:2605.23500#pg19>. The method establishes state-of-the-art performance in camouflage detection and remote sensing segmentation while providing strong gains on challenging reasoning segmentation <ref:2605. p. 823500#pg19>.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Bootstrapped Group Relative Tool Optimization (B-GRTO) can be used to jointly optimize a reasoning policy and a differentiable tool by first cheaply pre-aligning the tool using bootstrapped tool optimization (BTO). This leads to faster convergence and superior performance compared to plain GRPO, matching or surpassing domain-specific state-of-the-art results across challenging referring segmentation settings.

  2. The BTO stage allows for tool pretraining not hindered by policy rollout cost, making it a practical bottleneck solution for matching training velocities when the tool is lightweight but needs refinement before joint optimization. This ensures the tool is pre-aligned to the theoretical post-trained policy before the main GRTO stage begins.

  3. The joint optimization objective, JGRTO, maximizes a combined objective that integrates policy gradient updates with an auxiliary tool gradient update, allowing both the reasoning policy and the segmentation decoder to adapt simultaneously during training. This aligns the tool with the improved reasoning strategy that reinforcement learning is expected to produce.

  4. The system can achieve superior performance in camouflage detection and remote sensing segmentation by treating the segmentation decoder as a trainable component rather than a fixed reward component. This fine-tuning, when aligned with the policy objective, leads to masks that are better delineated and adapt to dataset-specific annotation semantics.

  5. The framework enables robust reasoning in challenging scenarios where both prompt disambiguation and domain adaptation are required simultaneously. B-GRTO demonstrates superior performance in challenging reasoning segmentation compared to policy-only GRPO, achieving the highest number of first-place rankings among evaluated configurations.

Sources

Related papers