ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning

arXiv:2605.20342 · cs.CV · Submitted 2026-05-19 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning".

Jane: Here's the gist of the paper. Training large multimodal models (LMMs) via reinforcement learning (RL) to natively invoke video-processing tools has become a promising route to long-video understanding However, existing native-RL methods dispatch tool calls sequentially (i.e., one per turn): a single wrong crop propagates errors without peer correction, multi-turn tool calls corrupt context, and inference cost scales linearly with the number of turns We introduce ParaVT, the first multi-agent end-to-end RL-trained framework for Parallel Video Tool calling, dispatching multiple time-window crops in a single turn for cleaner context and better fault tolerance Yet applying standard RL to ParaVT reveals an obstacle we term the Tool Prior Paradox: the pretrained tool priors that enable tool exploration also destabilize cold-started structural format and expose the skip-tool reward shortcut under temperature sampling We propose PARA-GRPO (Parseability-Anchored and Ratio-gAted GRPO), which augments standard RL with two complementary mechanisms: (i) a targeted format reward applied only at the structural-token positions most prone to collapse, and (ii) a per-prompt frame-budget randomization that creates training prompts where calling the tool yields a measurable reward signal over skipping it Across six longvideo understanding benchmarks, ParaVT improves over the Qwen3-VL baseline by +7.9% on average, with PARA-GRPO lifting training-time format compliance from 0.13 to 0.64 Pruning redundant reasoning tokens The paper identifies the Tool Prior Paradox, which is decomposed into Format Fragility and Tool Necessity Gap Format Fragility manifests in two forms: tag-level reversion (i.e., the policy emitting the pretrained schema in place of) and structural-boundary collapse (i.e., failure to close and).

Tom: The authors verify this diagnosis with a cross-model contrast on a weak-prior LMM, noting that format stays stable but RL elicits zero tool calls, indicating that prior strength is the shared driver of both format collapse and tool exploration.

Jane: How the training signal is built The PARA-GRPO framework introduces two complementary mechanisms to tame the paradox: Exploration Anchoring and nFrames Gating.

Tom: Exploration Anchoring stabilizes the format side via two cooperating mechanisms: a selective reward term targets the few structural-token positions most vulnerable to collapse, and a Constrained Generation hook fixes only the opening reasoning tag.

Paper summary: Tom: So we've talked about the main idea of ParaVT, and now let's look a bit closer at what they actually claim in this paper. The core thesis revolves around moving away from sequential tool calling to parallel tool calling in one step for long videos.

Jane: They are specifically targeting the problem where a single wrong crop error gets passed along without peer correction, and multi-turn calls just mess up the context structure of the model.

Lu: ParaVT’s approach is dispatching multiple time window crops in a single turn, which they claim leads to cleaner context and better fault tolerance compared to traditional sequential methods.

Meng: Cleaner context sounds good for accuracy, but I'm curious about how this parallel dispatch actually impacts inference cost when you have many tools being called simultaneously.

Lalam: The paper also introduces the Tool Prior Paradox, which is the central challenge they identified: pretrained tool priors that enable exploration can destabilize the cold-started structural format and expose a shortcut where skipping tools becomes rewarded under temperature sampling.

Tom: That paradox is tricky because it suggests that the very thing we use to get models to try new things—those pretraining priors—is what causes them to break their basic rules when they start learning via reinforcement learning.

Jane: To fight this, they propose PARA-GRPO, which uses Exploration Anchoring and nFrames Gating. The anchoring part fixes format issues by selectively rewarding structural tokens most prone to collapse, and the gating part creates a curriculum for training by randomizing the overview frame budget per prompt so tools are called when necessary.

Lu: The exploration anchoring mechanism is designed to stabilize the format side by using a selective reward term on those vulnerable structural token positions, and it also includes a constrained generation hook to fix only the opening reasoning tag.

Meng: That sounds like they are trying to keep the rollouts well-formed enough so they can actually be parsed into tool calls, which is crucial for any RL training pipeline.

Lalam: And then there's the nFrames Gating, which tackles the reward signal by randomizing the overview frame budget per prompt, setting up a curriculum where calling a tool yields a measurable reward signal over skipping it.

Conclusion: Tom: So, to wrap up ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning, what’s the big picture here? It’s about moving from brittle sequential chains to a parallel dispatch system for video tools.

Jane: The authors are essentially showing that just having a model that can use tools isn't enough; you need a way to make those tool uses robust and context-aware in long videos.

Lu: They propose PARA-GRPO as the solution, which combines format anchoring and reward gating to tame the trade-off between exploration and structural integrity. It shows that you can use pretrained knowledge without it breaking the model during reinforcement learning training under temperature sampling.

Meng: From a deployment standpoint, if this works, it suggests we could build more reliable agents for video understanding where we don't have to worry about cascading errors from a single bad tool execution.

Lalam: The main implication is that you can get better performance on those long video benchmarks with models like ParaVT when they are trained with PARA-GRPO, achieving a seven point nine percent average gain over the Qwen3 VL baseline.

Tom: So, in short, it’s about parallel tool use for long videos. It moves us away from the slow, error-prone sequential way and introduces a structured training method to handle the inherent instability of pretrained priors during RL training.

Zuhao Yang, Kaichen Zhang, Sudong Wang, Keming Wu, Zhongyu Yang

MiroMind · NTU · HKU-HKUST(GZ) · THU

cs.CV

Submitted: 2026-05-19

Updated: 2026-11-04

Comments: NeurIPS 2026

Code: https://github.com/EvolvingLMMs-Lab/ParaVT

Project page: https://evolvinglmms-lab.github.io/ParaVT/Abstract

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 99/100

The gist: Training large multimodal models (LMMs) via reinforcement learning (RL) to natively invoke video-processing tools has become a promising route to long-video understanding However…

Key concepts

Tool Prior Paradox
This paradox describes how the knowledge from pre-trained tools—which helps the model explore new actions—simultaneously destabilizes the model's learned structural format (like correct tagging) and encourages it to skip necessary tool calls during reinforcement learning training.
Format Fragility
This refers to when a model, while exploring tools, reverts to its initial pre-trained structure instead of following the desired output format. This manifests as incorrect tags or failing to close necessary structural boundaries in the generated text.
Tool Necessity Gap
This occurs when the reward signal for calling a tool is too weak compared to skipping it, especially when a direct answer is possible. The model learns to exploit this gap by skipping tools, leading to poor task accuracy despite the presence of tool-use capabilities.

Terminology

Summary

Training large multimodal models (LMMs) via reinforcement learning (RL) to natively invoke video-processing tools has become a promising route to long-video understanding However, existing native-RL methods dispatch tool calls sequentially (i. p. 0

Where the paper says it

io/ParaVT/ Abstract Training large multimodal models (LMMs) via reinforcement learning (RL) to natively invoke video-processing tools (e. , cropping) has become a promising route to long-video understanding.

e., one per turn): a single wrong crop propagates errors without peer correction, multi-turn tool calls corrupt context, and inference cost scales linearly with the number of turns We introduce ParaVT, the first multi-agent end-to-end RL-trained framework for Parallel Video Tool calling, dispatching multiple time-window crops in a single turn for cleaner context and better fault tolerance Yet applying standard RL to ParaVT reveals an obstacle we term the Tool Prior Paradox: the pretrained tool priors that enable tool exploration also destabilize cold-started structural format and expose the skip-tool reward shortcut under temperature sampling We propose PARA-GRPO (Parseability-Anchored and Ratio-gAted GRPO), which augments standard RL with two complementary mechanisms: (i) a targeted format reward applied only at the structural-token positions most prone to collapse, and (ii) a per-prompt frame-budget randomization that creates training prompts where calling the tool yields a measurable reward signal over skipping it Across six longvideo understanding benchmarks, ParaVT improves over the Qwen3-VL baseline by +7. p. 0
Where the paper says it

We introduce ParaVT, the first multi-agent end-to-end RL-trained framework for Parallel Video Tool calling, dispatching multiple time-window crops in a single turn for cleaner context and better fault tolerance. Yet applying standard RL to ParaVT reveals an obstacle we term the Tool Prior Paradox: the pretrained tool priors that enable tool exploration also destabilize cold-started structural format and expose the skip-tool reward shortcut under temperature sampling.

p. 0
Where the paper says it

We introduce ParaVT, the first multi-agent end-to-end RL-trained framework for Parallel Video Tool calling, dispatching multiple time-window crops in a single turn for cleaner context and better fault tolerance. Yet applying standard RL to ParaVT reveals an obstacle we term the Tool Prior Paradox: the pretrained tool priors that enable tool exploration also destabilize cold-started structural format and expose the skip-tool reward shortcut under temperature sampling.

9% on average, with PARA-GRPO lifting training-time format compliance from 0. p. 0
Where the paper says it

Across six longvideo understanding benchmarks, ParaVT improves over the Qwen3-VL baseline by +7. 9% on average, with PARA-GRPO lifting training-time format compliance from 0.

p. 0
Where the paper says it

Across six longvideo understanding benchmarks, ParaVT improves over the Qwen3-VL baseline by +7. 9% on average, with PARA-GRPO lifting training-time format compliance from 0.

13 to 0.64

Pruning redundant reasoning tokens

The paper identifies the Tool Prior Paradox, which is decomposed into Format Fragility and Tool Necessity Gap Format Fragility manifests in two forms: tag-level reversion (i.e., the policy emitting the pretrained schema in place of) and structural-boundary collapse (i. p. 4

Where the paper says it

2 PARA-GRPO: Parseability-Anchored and Ratio-Gated GRPO Format Fragility manifests in two forms: tag-level reversion (i. , the policy emitting the pretrained <tool_code> schema in place of <tool_call>) and structural-boundary collapse (i.

e., failure to close and). p. 4
Where the paper says it

, the policy emitting the pretrained <tool_code> schema in place of <tool_call>) and structural-boundary collapse (i. , failure to close </think> and </answer>).

The authors verify this diagnosis with a cross-model contrast on a weak-prior LMM, noting that format stays stable but RL elicits zero tool calls, indicating that prior strength is the shared driver of both format collapse and tool exploration. p. 0
Where the paper says it

Yet applying standard RL to ParaVT reveals an obstacle we term the Tool Prior Paradox: the pretrained tool priors that enable tool exploration also destabilize cold-started structural format and expose the skip-tool reward shortcut under temperature sampling. A cross-model contrast on a weaker-prior LMM supports this claim: format stays stable but RL elicits zero tool calls, indicating that prior strength is the shared driver of both format collapse and tool exploration.

How the training signal is built

The PARA-GRPO framework introduces two complementary mechanisms to tame the paradox: Exploration Anchoring and nFrames Gating. Exploration Anchoring stabilizes the format side via two cooperating mechanisms: a selective reward term targets the few structural-token positions most vulnerable to collapse, and a Constrained Generation hook fixes only the opening reasoning tag. p. 2

Where the paper says it

We propose PARA-GRPO (Parseability-Anchored and Ratio-gAted GRPO) with Exploration Anchoring and nFrames Gating to tame the Tool Prior Paradox (Section 3. Exploration Anchoring stabilizes the format side via two cooperating mechanisms: a selective reward term targets the few structural-token positions most vulnerable to collapse, and a Constrained Generation hook fixes only the opening reasoning tag.

nFrames Gating tackles the reward-signal side by randomizing the overview-frame budget per prompt, creating a curriculum where calling the tool yields a measurable reward signal over skipping it. p. 2
Where the paper says it

Together they anchor rollout parseability without restricting reasoning content or tool-call sequences. nFrames Gating tackles the reward-signal side: randomizing the overview-frame budget per prompt creates a curriculum where a fraction of prompts cannot be answered from overview frames alone, gating a non-trivial call/skip advantage ratio that vanilla GRPO would otherwise average to zero.

The two design choices are complementary: anchoring keeps rollouts well-formed enough to be parseable, and only on parseable rollouts can gating credit the tool-reward gradient. p. 2
Where the paper says it

nFrames Gating tackles the reward-signal side: randomizing the overview-frame budget per prompt creates a curriculum where a fraction of prompts cannot be answered from overview frames alone, gating a non-trivial call/skip advantage ratio that vanilla GRPO would otherwise average to zero. The two design choices are complementary: anchoring keeps rollouts well-formed enough to be parseable, and only on parseable rollouts can gating credit the tool-reward gradient.

Where the gains come from

The empirical results demonstrate that ParaVT outperforms all comparable open-source 7–8B baselines on six of seven evaluation splits, with the largest gains concentrated on the long-video MCQ subset (+15.7% on LongVideoBench, +20. p. 6

Where the paper says it

Evaluation. We evaluate on six long-video benchmarks under a unified 64-frame adaptive protocol, reporting MCQ accuracy on VideoMME [8], LongVideoBench [38], LVBench [36], MLVU [53], and MMVU [51], and mean Intersection over Union (mIoU) on Charades-STA [11].

2% on LVBench, +11. p. 6
Where the paper says it

Evaluation. We evaluate on six long-video benchmarks under a unified 64-frame adaptive protocol, reporting MCQ accuracy on VideoMME [8], LongVideoBench [38], LVBench [36], MLVU [53], and MMVU [51], and mean Intersection over Union (mIoU) on Charades-STA [11].

5% on MLVU; an average relative gain of +7. p. 6
Where the paper says it

2% on LVBench, +11. 5% on MLVU; an average relative gain of +7.

9% across all seven splits). p. 6
Where the paper says it

2 Main Results As shown in Table 1, ParaVT outperforms all comparable open-source 7–8B baselines on six of the seven evaluation splits. ParaVT improves on its Qwen3-VL-8B base model across every split, with the largest gains concentrated on the long-video MCQ subset (+15.

The full PARA-GRPO recipe stabilizes both axes, reaching the highest training-time mean format reward in Block A (fτ=0. p. 8
Where the paper says it

13 while κ collapses to near zero; Exploration Anchoring (orange) lifts fτ but keeps κ moderate; nFrames Gating (green) pushes κ off-chart while leaving fτ low; only the full PARAGRPO (blue) stabilizes both axes. PARA-GRPO resolves both extremes, reaching the highest training-time mean format reward in Block A (fτ=0.

41, κ=0.21), and strictly improves on vanilla GRPO across all six evaluation splits. p. 8
Where the paper says it

PARA-GRPO resolves both extremes, reaching the highest training-time mean format reward in Block A (fτ=0. 21), and strictly improves on vanilla GRPO across all six evaluation splits, with the largest gains on LongVideoBench and MMVU.

Dispatch Mode

The parallel dispatch architecture of ParaVT re-organizes the tool-using loop as a single-turn divide-and-conquer step, dispatching K parallel invocations on disjoint temporal windows to K independent sub-agents concurrently. This yields three concrete advantages over the sequential paradigm: (i) Peer-Correctable Evidence, where a mis-localized window is outvoted by its peers rather than propagated down a serial chain; (ii) Controlled Context Growth, as returning text summaries adds only a small constant extension to the running context, while returning original frames would re-inflate it with K visual-token blocks per turn; and (iii) Bounded Inference Latency, as the K sub-agents run concurrently so the tool-using portion of the rollout is bounded by the slowest sub-agent rather than by their sum. p. 3

Where the paper says it

Returning text summaries adds only a small constant extension to the running context, while returning original frames would re-inflate it with K visual-token blocks per turn. (iii) Bounded Inference Latency.

Conclusion

In this work, we present ParaVT, the first multi-agent end-to-end RL-trained framework that enables tool-native LMMs to dispatch Parallel Video Tool calls in a single turn for long-video reasoning, replacing brittle sequential tool chains with peer-correctable evidence aggregation while keeping inference cost flat as the number of dispatched tools grows. By identifying the central training trade-off as the Tool Prior Paradox (the dual role of pretrained tool priors in driving both tool exploration and structural-format collapse under temperature sampling), we propose PARA-GRPO, which augments standard GRPO with a parseability-anchored format reward applied only at the structural-token positions most prone to collapse, and a ratio-gated frame-budget randomization that credits tools only on prompts where they are genuinely necessary. p. 8

Where the paper says it

5 Conclusion In this work, we present ParaVT, the first multi-agent end-to-end RL-trained framework that enables tool-native LMMs to dispatch Parallel Video Tool calls in a single turn for long-video reasoning, replacing brittle sequential tool chains with peer-correctable evidence aggregation while keeping inference cost flat as the number of dispatched tools grows. By identifying the central training trade-off as the Tool Prior Paradox (the dual role of pretrained tool priors in driving both tool exploration and structural-format collapse under temperature sampling), we propose PARA-GRPO, which augments standard GRPO with a parseability-anchored format reward applied only at the structural-token positions most prone to collapse, and a ratio-gated frame-budget randomization that credits tools only on prompts where they are genuinely necessary.

Supported by a self-curated 97K-sample multi-task SFT corpus and a separate 4,406-sample RL split spanning open-ended QA, multiple-choice, and temporal grounding, ParaVT outperforms existing open-source 7–8B baselines on six of seven long-video evaluation splits. p. 9
Where the paper says it

97K-sample multi-task SFT corpus and a separate 4,406-sample RL split spanning open-ended QA, multiple-choice, and temporal grounding, ParaVT outperforms existing open-source 7–8B baselines on six of seven long-video evaluation splits, demonstrating that anchoring format and gating tool incentives is a transferable recipe for agentic RL as tool capabilities become increasingly internalized in modern base LMMs. References [1] Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al.

--- Page 1 ---

ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning Zuhao Yang2,6, Kaichen Zhang3,6, Sudong Wang4, Keming Wu5,6, Zhongyu Yang2 Bo Li6 Xiaojuan Qi3 Shijian Lu2B Xingxuan Li1Lidong Bing1 1MiroMind 2NTU 3HKU 4HKUST(GZ) 5THU 6LMMs-Lab https://evolvinglmms-lab.github.io/ParaVT/

Abstract Training large multimodal models (LMMs) via reinforcement learning (RL) to natively invoke video-processing tools (e.g., cropping) has become a promising route to long-video understanding However, existing native-RL methods dispatch tool calls sequentially (i. p. 0

Where the paper says it

, cropping) has become a promising route to long-video understanding. However, existing native-RL methods dispatch tool calls sequentially (i.

e., one per turn): a single wrong crop propagates errors without peer correction, multi-turn tool calls corrupt context, and inference cost scales linearly with the number of turns We introduce ParaVT, the first multi-agent end-to-end RL-trained framework for Parallel Video Tool calling, dispatching multiple time-window crops in a single turn for cleaner context and better fault tolerance Yet applying standard RL to ParaVT reveals an obstacle we term the Tool Prior Paradox: the pretrained tool priors that enable tool exploration also destabilize cold-started structural format and expose the skip-tool reward shortcut under temperature sampling We propose PARA-GRPO (Parseability-Anchored and Ratio-gAted GRPO), which augments standard RL with two complementary mechanisms: (i) a targeted format reward applied only at the structural-token positions most prone to collapse, and (ii) a per-prompt frame-budget randomization that creates training prompts where calling the tool yields a measurable reward signal over skipping it Across six longvideo understanding benchmarks, ParaVT improves over the Qwen3-VL baseline by +7.9% on average, with PARA-GRPO lifting training-time format compliance from 0.13 to 0.64

--- Page 2 ---

Greedy Decoding under Evaluation = 0; f = 1 (when correct) Closer look.crop(0:42-0:48) B well-formed Closer look. p. 1

Where the paper says it

Greedy Decoding under Evaluation = 0; f = 1 (when correct) <think>Closer look. </think> <tool_call>crop(0:42-0:48) </tool_call> <answer> B </answer> well-formed <think> / <tool_call> / <answer> structure Temperature Sampling under GRPO = 0.

print(crop video(in place of; closing tag missing; no × (a) Format Fragility 0 1 2 3 4 5 6 7 8 RL Step =0. p. 1
Where the paper says it

</think> <tool_code> print(crop_video(. )) <tool_code> in place of <tool_call>; closing tag missing; no <answer> × (a) Format Fragility 0 1 2 3 4 5 6 7 8 9 RL Step 0.

05 (b) Tool Necessity Gap Tool Calls / Sample Task Accuracy (smoothed) Task Accuracy (raw) =0. p. 1
Where the paper says it

)) <tool_code> in place of <tool_call>; closing tag missing; no <answer> × (a) Format Fragility 0 1 2 3 4 5 6 7 8 9 RL Step 0. 25 (b) Tool Necessity Gap Tool Calls / Sample Task Accuracy (smoothed) Task Accuracy (raw) 0.

p. 1
Where the paper says it

)) <tool_code> in place of <tool_call>; closing tag missing; no <answer> × (a) Format Fragility 0 1 2 3 4 5 6 7 8 9 RL Step 0. 25 (b) Tool Necessity Gap Tool Calls / Sample Task Accuracy (smoothed) Task Accuracy (raw) 0.

0,0.2,0.4,0.6,0.8 Figure 1: Two Failure Modes of the Tool Prior Paradox. p. 1
Where the paper says it

25 (b) Tool Necessity Gap Tool Calls / Sample Task Accuracy (smoothed) Task Accuracy (raw) 0. 8 Figure 1: Two Failure Modes of the Tool Prior Paradox.

p. 1
Where the paper says it

25 (b) Tool Necessity Gap Tool Calls / Sample Task Accuracy (smoothed) Task Accuracy (raw) 0. 8 Figure 1: Two Failure Modes of the Tool Prior Paradox.

(a) Format Fragility: rollouts are wellformed under greedy decoding (sampling temperature τ=0, format reward ≈ 1); under temperature sampling within vanilla GRPO (τ=0. p. 1
Where the paper says it

8 Figure 1: Two Failure Modes of the Tool Prior Paradox. (a) Format Fragility: rollouts are wellformed under greedy decoding (sampling temperature τ=0, format reward ≈ 1); under temperature sampling within vanilla GRPO (τ=0.

p. 1
Where the paper says it

8 Figure 1: Two Failure Modes of the Tool Prior Paradox. (a) Format Fragility: rollouts are wellformed under greedy decoding (sampling temperature τ=0, format reward ≈ 1); under temperature sampling within vanilla GRPO (τ=0.

7), the policy reverts to the pretrained tag in place of, often drops closing tags, and stops emitting altogether (fτ≈0. p. 1
Where the paper says it

(a) Format Fragility: rollouts are wellformed under greedy decoding (sampling temperature τ=0, format reward ≈ 1); under temperature sampling within vanilla GRPO (τ=0. 7), the policy reverts to the pretrained <tool_code> tag in place of <tool_call>, often drops closing tags, and stops emitting <answer> altogether (fτ≈0.

p. 1
Where the paper says it

(a) Format Fragility: rollouts are wellformed under greedy decoding (sampling temperature τ=0, format reward ≈ 1); under temperature sampling within vanilla GRPO (τ=0. 7), the policy reverts to the pretrained <tool_code> tag in place of <tool_call>, often drops closing tags, and stops emitting <answer> altogether (fτ≈0.

1) (b) Tool Necessity Gap: tool-call count drops to near-zero within 7 steps while task accuracy oscillates between 0. p. 1
Where the paper says it

7), the policy reverts to the pretrained <tool_code> tag in place of <tool_call>, often drops closing tags, and stops emitting <answer> altogether (fτ≈0. (b) Tool Necessity Gap: tool-call count drops to near-zero within 7 steps while task accuracy oscillates between 0.

p. 1
Where the paper says it

7), the policy reverts to the pretrained <tool_code> tag in place of <tool_call>, often drops closing tags, and stops emitting <answer> altogether (fτ≈0. (b) Tool Necessity Gap: tool-call count drops to near-zero within 7 steps while task accuracy oscillates between 0.

45 and 0.74, as the policy converges on the shortcut of skipping tools Figure 2 Cross-Model Evidence for the Tool Prior Paradox. p. 1
Where the paper says it

As a result, malformed rollouts cannot be parsed into rewardable tool calls, so the GRPO advantage signal is computed over a corrupted trajectory population before any tool-use credit can be assigned. The second is Tool Necessity Gap (Figure 1b): when uniformly-sampled overview frames suffice to answer many prompts directly, the reward gap between “call tool” and “skip tool” rollouts is near-zero, so GRPO’s group-normalized advantage on the call/skip dimension is also near-zero, and the policy converges to the canonical reward-hacking shortcut of skipping tools [29].

p. 1
Where the paper says it

As a result, malformed rollouts cannot be parsed into rewardable tool calls, so the GRPO advantage signal is computed over a corrupted trajectory population before any tool-use credit can be assigned. The second is Tool Necessity Gap (Figure 1b): when uniformly-sampled overview frames suffice to answer many prompts directly, the reward gap between “call tool” and “skip tool” rollouts is near-zero, so GRPO’s group-normalized advantage on the call/skip dimension is also near-zero, and the policy converges to the canonical reward-hacking shortcut of skipping tools [29].

Qwen3-VL-8B (stronger tool prior) explores tool use but collapses on format, while Qwen2. p. 1
Where the paper says it

5-VL (weak prior) Qwen3-VL (strong prior) Figure 2: Cross-Model Evidence for the Tool Prior Paradox under Vanilla GRPO. Qwen3- VL-8B (stronger tool prior) explores tool use but collapses on format, while Qwen2.

p. 1
Where the paper says it

5-VL (weak prior) Qwen3-VL (strong prior) Figure 2: Cross-Model Evidence for the Tool Prior Paradox under Vanilla GRPO. Qwen3- VL-8B (stronger tool prior) explores tool use but collapses on format, while Qwen2.

5-VL-7B (weaker tool prior) stays format-perfect yet emits zero tool calls

--- Page 3 ---

A natural choice for end-to-end ParaVT training is Group Relative Policy Optimization (GRPO) [12] on top of a tool-native coldstarted Qwen3-VL [1] checkpoint However, vanilla GRPO exhibits two coupled training-time failures. The first is Format Fragility (Figure 1a): the SFT-learned // format is reliable under greedy decoding but, within a few vanilla-GRPO steps under temperature sampling, the policy reverts to the pretrained schema. p. 1

Where the paper says it

However, vanilla GRPO exhibits two coupled training-time failures. The first is Format Fragility (Figure 1a): the SFT-learned <think>/<tool_call>/<answer> format is reliable under greedy decoding but, within a few vanilla-GRPO steps under temperature sampling, the policy reverts to the pretrained <tool_code> schema.

p. 1
Where the paper says it

However, vanilla GRPO exhibits two coupled training-time failures. The first is Format Fragility (Figure 1a): the SFT-learned <think>/<tool_call>/<answer> format is reliable under greedy decoding but, within a few vanilla-GRPO steps under temperature sampling, the policy reverts to the pretrained <tool_code> schema.

p. 1
Where the paper says it

However, vanilla GRPO exhibits two coupled training-time failures. The first is Format Fragility (Figure 1a): the SFT-learned <think>/<tool_call>/<answer> format is reliable under greedy decoding but, within a few vanilla-GRPO steps under temperature sampling, the policy reverts to the pretrained <tool_code> schema.

This is a shallow override of the SFT format reminiscent of the Superficial Alignment Hypothesis [52], compounded by the competing pretrained tool priors: the probability mass on tool-call continuations carried over from pretraining (before SFT) that resurfaces under RL-time temperature. p. 1
Where the paper says it

The first is Format Fragility (Figure 1a): the SFT-learned <think>/<tool_call>/<answer> format is reliable under greedy decoding but, within a few vanilla-GRPO steps under temperature sampling, the policy reverts to the pretrained <tool_code> schema. This is a shallow override of the SFT format reminiscent of the Superficial Alignment Hypothesis [52], compounded by the competing pretrained tool priors: the probability mass on tool-call continuations carried over from pretraining (before SFT) that resurfaces under RL-time temperature.

p. 1
Where the paper says it

The first is Format Fragility (Figure 1a): the SFT-learned <think>/<tool_call>/<answer> format is reliable under greedy decoding but, within a few vanilla-GRPO steps under temperature sampling, the policy reverts to the pretrained <tool_code> schema. This is a shallow override of the SFT format reminiscent of the Superficial Alignment Hypothesis [52], compounded by the competing pretrained tool priors: the probability mass on tool-call continuations carried over from pretraining (before SFT) that resurfaces under RL-time temperature.

p. 1
Where the paper says it

The first is Format Fragility (Figure 1a): the SFT-learned <think>/<tool_call>/<answer> format is reliable under greedy decoding but, within a few vanilla-GRPO steps under temperature sampling, the policy reverts to the pretrained <tool_code> schema. This is a shallow override of the SFT format reminiscent of the Superficial Alignment Hypothesis [52], compounded by the competing pretrained tool priors: the probability mass on tool-call continuations carried over from pretraining (before SFT) that resurfaces under RL-time temperature.

As a result, malformed rollouts cannot be parsed into rewardable tool calls, so the GRPO advantage signal is computed over a corrupted trajectory population before any tool-use credit can be assigned The second is Tool Necessity Gap (Figure 1b): when uniformly-sampled overview frames suffice to answer many prompts directly, the reward gap between “call tool” and “skip tool” rollouts is near-zero, so GRPO’s group-normalized advantage on the call/skip dimension is also near-zero, and the policy converges to the canonical reward-hacking shortcut of skipping tools To probe the role of pretrained tool priors, we replicate the same setup on Qwen2. p. 1
Where the paper says it

As a result, malformed rollouts cannot be parsed into rewardable tool calls, so the GRPO advantage signal is computed over a corrupted trajectory population before any tool-use credit can be assigned. The second is Tool Necessity Gap (Figure 1b): when uniformly-sampled overview frames suffice to answer many prompts directly, the reward gap between “call tool” and “skip tool” rollouts is near-zero, so GRPO’s group-normalized advantage on the call/skip dimension is also near-zero, and the policy converges to the canonical reward-hacking shortcut of skipping tools [29].

p. 1
Where the paper says it

As a result, malformed rollouts cannot be parsed into rewardable tool calls, so the GRPO advantage signal is computed over a corrupted trajectory population before any tool-use credit can be assigned. The second is Tool Necessity Gap (Figure 1b): when uniformly-sampled overview frames suffice to answer many prompts directly, the reward gap between “call tool” and “skip tool” rollouts is near-zero, so GRPO’s group-normalized advantage on the call/skip dimension is also near-zero, and the policy converges to the canonical reward-hacking shortcut of skipping tools [29].

5-VL [25] (with much weaker tool priors than Qwen3-VL) under identical hyperparameters (Figure 2): its format stays near-perfect yet RL elicits no tool calls A natural choice for end-to-end ParaVT training is Group Relative Policy Optimization (GRPO) [12] on top of a tool-native coldstarted Qwen3-VL [1] checkpoint

--- Page 4 ---

Figure 3 Framework Comparison.(a) Sequential Tool Calling: successive turns re-include the full context, accumulating visual-token overhead; a single mis-localized crop propagates errors with no peer to correct it, yielding an error-amplified answer. p. 3

Where the paper says it

(a) Sequential Tool Calling Video Frames + User Query T1: crop(0:10-1:20) +5K tokens T2: crop(2:00-2:50) +5K tokens T3: crop(3:30-4:30) +5K tokens Error-Amplified Answer visual-text ratio saturates per turn (b) Parallel Tool Calling (Ours) Video Frames + User Query Main Agent: Plan & Dispatch Sub-Agent 1 (0:10-1:20) Sub-Agent 2 (2:00-2:50) Sub-Agent 3 (3:30-4:30) concurrent execution Main Agent: Gather & Reason Evidence-Aggregated Answer Figure 3: Framework Comparison. (a) Sequential Tool Calling: successive turns re-include the full context, accumulating visual-token overhead; a single mis-localized crop (✗) propagates errors with no peer to correct, yielding an error-amplified answer.

(b) Parallel Tool Calling (Ours): one main agent dispatches K tool calls concurrently to K independent sub-agents that share weights with the main agent. p. 3
Where the paper says it

(a) Sequential Tool Calling: successive turns re-include the full context, accumulating visual-token overhead; a single mis-localized crop (✗) propagates errors with no peer to correct, yielding an error-amplified answer. (b) Parallel Tool Calling (Ours): one main agent dispatches K tool calls concurrently to K independent sub-agents (shown for K=3); mis-localized peers (✗) are outvoted by correct ones (✓), yielding an evidence-aggregated answer.

Each sub-agent grounds only its assigned window, samples a short crop, and returns a textual summary rather than resampled frames. p. 3
Where the paper says it

Within a single turn, the main agent emits K parallel <tool_call> invocations on disjoint temporal windows, each dispatched to one of K independent sub-agents that share weights with the main agent. Each sub-agent grounds only its assigned window, samples a short crop, and returns a textual summary rather than resampled frames.

p. 3
Where the paper says it

Within a single turn, the main agent emits K parallel <tool_call> invocations on disjoint temporal windows, each dispatched to one of K independent sub-agents that share weights with the main agent. Each sub-agent grounds only its assigned window, samples a short crop, and returns a textual summary rather than resampled frames.

The gathered summaries are concatenated into a single block on which the main agent reasons to generate the final

--- Page 5 ---

In ParaVT consists of three design elements: a parallel-dispatch architecture (Section 3.1.1), a two-stage training pipeline (Section 3. p. 3

Where the paper says it

1 ParaVT: Parallel Video Tool Calling for Long-Video Understanding ParaVT consists of three design elements: a parallel-dispatch architecture (Section 3. 1), a two-stage training pipeline (Section 3.

1.2), and a self-curated multi-task dataset (Section 3. p. 3
Where the paper says it

1), a two-stage training pipeline (Section 3. 2), and a self-curated multi-task dataset (Section 3.

1.3). A common paradigm for tool-augmented long-video understanding lets the LMM decide when and where in the video to look more closely by issuing a crop video(start, end) function call that returns the requested temporal segment with densely resampled frames for further inspection (Figure 3a). p. 3
Where the paper says it

2), and a self-curated multi-task dataset (Section 3. 1 Framework Design A common paradigm for tool-augmented long-video understanding lets the LMM decide when and where in the video to look more closely by issuing a crop_video(start, end) function call that returns the requested temporal segment with densely resampled frames for further inspection (Figure 3a).

ParaVT re-organizes the same loop as a single-turn divide-and-conquer step (Figure 3b) Within a single turn, the main agent emits K parallel invocations on disjoint temporal windows, each dispatched to one of K independent sub-agents that share weights with the main agent. p. 3
Where the paper says it

ParaVT re-organizes the same loop as a single-turn divide-and-conquer step (Figure 3b). Within a single turn, the main agent emits K parallel <tool_call> invocations on disjoint temporal windows, each dispatched to one of K independent sub-agents that share weights with the main agent.

Each sub-agent grounds only its assigned window, samples a short crop, and returns a textual summary rather than resampled frames. The gathered summaries are concatenated into a single block on which the main agent reasons to generate the final

--- Page 6 ---

This single-turn parallel dispatch yields three concrete advantages over the sequential paradigm: (i) Peer-Correctable Evidence, where the main agent receives K cross-checkable summaries grounded in disjoint windows by independent sub-agents, so a mis-localized window is outvoted by its peers rather than propagated down a serial chain; (ii) Controlled Context Growth, as returning text summaries adds only a small constant extension to the running context, while returning original frames would re-inflate it with K visual-token blocks per turn; and (iii) Bounded Inference Latency, as the K sub-agents run concurrently so the tool-using portion of the rollout is bounded by the slowest sub-agent rather than by their sum.

--- Page 7 ---

RL Step 0.05 (b) Tool Necessity Gap Tool Calls / Sample Task Accuracy (smoothed) Task Accuracy (raw) =0.0,0.2,0.4,0.6,0.8 Figure 1: Two Failure Modes of the Tool Prior Paradox.(a) Format Fragility: rollouts are wellformed under greedy decoding (sampling temperature τ=0, format reward ≈ 1); under temperature sampling within vanilla GRPO (τ=0.7), the policy reverts to the pretrained tag in place of, often drops closing tags, and stops emitting altogether (fτ≈0.1) (b) Tool Necessity Gap: tool-call count drops to near-zero within 7 steps while task accuracy oscillates between 0.45 and 0.74, as the policy converges on the shortcut of skipping tools Figure 2 Cross-Model Evidence for the Tool Prior Paradox. Qwen3-VL-8B (stronger tool prior) explores tool use but collapses on format, while Qwen2.5-VL-7B (weaker tool prior) stays format-perfect yet emits zero tool calls

--- Page 8 ---

A natural choice for end-to-end ParaVT training is Group Relative Policy Optimization (GRPO) [12] on top of a tool-native coldstarted Qwen3-VL [1] checkpoint However, vanilla GRPO exhibits two coupled training-time failures. The first is Format Fragility (Figure 1a): the SFT-learned // format is reliable under greedy decoding but, within a few vanilla-GRPO steps under temperature sampling, the policy reverts to the pretrained schema. This is a shallow override of the SFT format reminiscent of the Superficial Alignment Hypothesis [52], compounded by the competing pretrained tool priors: the probability mass on tool-call continuations carried over from pretraining (before SFT) that resurfaces under RL-time temperature. As a result, malformed rollouts cannot be parsed into rewardable tool calls, so the GRPO advantage signal is computed over a corrupted trajectory population before any tool-use credit can be assigned The second is Tool Necessity Gap (Figure 1b): when uniformly-sampled overview frames suffice to answer many prompts directly, the reward gap between “call tool” and “skip tool” rollouts is near-zero, so GRPO’s group-normalized advantage on the call/skip dimension is also near-zero, and the policy converges to the canonical reward-hacking shortcut of skipping tools To probe the role of pretrained tool priors, we replicate the same setup on Qwen2.5-VL [25] (with much weaker tool priors than Qwen3-VL) under identical hyperparameters (Figure 2): its format stays near-perfect yet RL elicits no tool calls

--- Page 9 ---

To probe the role of pretrained tool priors, we replicate the same setup on Qwen2.5-VL [25] (with much weaker tool priors than Qwen3-VL) under identical hyperparameters (Figure 2): its format stays near-perfect yet RL elicits no tool calls A natural choice for end-to-end ParaVT training is Group Relative Policy Optimization (GRPO) [12] on top of a tool-native coldstarted Qwen3-VL [1] checkpoint However, vanilla GRPO exhibits two coupled training-time failures. p. 1

Where the paper says it

A natural choice for end-to-end ParaVT training is Group Relative Policy Optimization (GRPO) [12] on top of a tool-native coldstarted Qwen3-VL [1] checkpoint. However, vanilla GRPO exhibits two coupled training-time failures.

The first is Format Fragility (Figure 1a): the SFT-learned // format is reliable under greedy decoding but, within a few vanilla-GRPO steps under temperature sampling, the policy reverts to the pretrained schema. This is a shallow override of the SFT format reminiscent of the Superficial Alignment Hypothesis [52], compounded by the competing pretrained tool priors: the probability mass on tool-call continuations carried over from pretraining (before SFT) that resurfaces under RL-time temperature. As a result, malformed rollouts cannot be parsed into rewardable tool calls, so the GRPO advantage signal is computed over a corrupted trajectory population before any tool-use credit can be assigned The second is Tool Necessity Gap (Figure 1b): when uniformly-sampled overview frames suffice to answer many prompts directly, the reward gap between “call tool” and “skip tool” rollouts is near-zero, so GRPO’s group-normalized advantage on the call/skip dimension is also near-zero,… p. 1
Where the paper says it

As a result, malformed rollouts cannot be parsed into rewardable tool calls, so the GRPO advantage signal is computed over a corrupted trajectory population before any tool-use credit can be assigned. The second is Tool Necessity Gap (Figure 1b): when uniformly-sampled overview frames suffice to answer many prompts directly, the reward gap between “call tool” and “skip tool” rollouts is near-zero, so GRPO’s group-normalized advantage on the call/skip dimension is also near-zero, and the policy converges to the canonical reward-hacking shortcut of skipping tools [29].

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Parallel Video Tool Calling via ParaVT enables a main agent to issue multiple time-window crops in a single turn for cleaner context and better fault tolerance, replacing brittle sequential tool chains with peer-correctable evidence aggregation.

  2. PARA-GRPO (Parseability-Anchored and Ratio-gated GRPO) augments standard RL with two complementary mechanisms: (i) a targeted format reward applied only at the structural-token positions most prone to collapse, and (ii) a per-prompt frame-budget randomization that creates training prompts where calling the tool yields a measurable reward signal over skipping it.

  3. Exploration Anchoring repairs rollout parseability at the structural-token boundaries where collapse concentrates by using a selective reward term that targets specific token positions, while nFrames Gating creates the missing gap by randomizing the overview-frame budget to generate a non-trivial reward contrast between tool-calling and tool-skipping rollouts.

  4. The improved system can achieve better performance on long video understanding benchmarks, such as LongVideoBench (60.4) and MMVU (68.6), demonstrating that anchoring format and gating tool incentives is a transferable recipe for agentic RL.

Abstract

Training large multimodal models (LMMs) via reinforcement learning (RL) to natively invoke video-processing tools (e.g., cropping) has become a promising route to long-video understanding. However, most existing agentic Video-LMMs dispatch tool calls sequentially (i.e., one per turn). Sequential tool calling can propagate localization errors, accumulate irrelevant context, and require an additional main-model generation for each successive crop. We introduce ParaVT, a multi-agent RL framework for Parallel Video Tool calling, dispatching multiple time-window crops in a single turn, allowing evidence from multiple temporal windows to be examined concurrently. Yet applying standard Group Relative Policy Optimization (GRPO) to ParaVT reveals an obstacle we term the Tool Prior Paradox: inherited tool-use priors appear to support exploration while competing with the output format learned during supervised fine-tuning. Standard RL can also settle on tool skipping. A comparison with a model with weaker tool-use priors shows stable formatting but no tool calls during RL, consistent with a trade-off between format stability and tool exploration. We propose PARA-GRPO (Parseability-Anchored and Ratio-gAted GRPO), which augments standard RL with two complementary mechanisms: (i) a format reward that encourages tag closure and penalizes unclosed reasoning blocks, and (ii) a per-prompt randomization of the overview-frame budget to create a reward advantage for tool use when the overview provides insufficient evidence. Across six long-video understanding benchmarks, ParaVT improves over the Qwen3-VL baseline by +7.9% on average, with PARA-GRPO raising training-time format reward from 0.13 to a peak of 0.64. These findings highlight the need to account for inherited tool-use priors when designing RL methods for long-video understanding.

Sources

Related papers