ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning".
Jane: Here's the gist of the paper. Training large multimodal models (LMMs) via reinforcement learning (RL) to natively invoke video-processing tools has become a promising route to long-video understanding However, existing native-RL methods dispatch tool calls sequentially (i.e., one per turn): a single wrong crop propagates errors without peer correction, multi-turn tool calls corrupt context, and inference cost scales linearly with the number of turns We introduce ParaVT, the first multi-agent end-to-end RL-trained framework for Parallel Video Tool calling, dispatching multiple time-window crops in a single turn for cleaner context and better fault tolerance Yet applying standard RL to ParaVT reveals an obstacle we term the Tool Prior Paradox: the pretrained tool priors that enable tool exploration also destabilize cold-started structural format and expose the skip-tool reward shortcut under temperature sampling We propose PARA-GRPO (Parseability-Anchored and Ratio-gAted GRPO), which augments standard RL with two complementary mechanisms: (i) a targeted format reward applied only at the structural-token positions most prone to collapse, and (ii) a per-prompt frame-budget randomization that creates training prompts where calling the tool yields a measurable reward signal over skipping it Across six longvideo understanding benchmarks, ParaVT improves over the Qwen3-VL baseline by +7.9% on average, with PARA-GRPO lifting training-time format compliance from 0.13 to 0.64 Pruning redundant reasoning tokens The paper identifies the Tool Prior Paradox, which is decomposed into Format Fragility and Tool Necessity Gap Format Fragility manifests in two forms: tag-level reversion (i.e., the policy emitting the pretrained schema in place of) and structural-boundary collapse (i.e., failure to close and).
Tom: The authors verify this diagnosis with a cross-model contrast on a weak-prior LMM, noting that format stays stable but RL elicits zero tool calls, indicating that prior strength is the shared driver of both format collapse and tool exploration.
Jane: How the training signal is built The PARA-GRPO framework introduces two complementary mechanisms to tame the paradox: Exploration Anchoring and nFrames Gating.
Tom: Exploration Anchoring stabilizes the format side via two cooperating mechanisms: a selective reward term targets the few structural-token positions most vulnerable to collapse, and a Constrained Generation hook fixes only the opening reasoning tag.
Paper summary: Tom: So we've talked about the main idea of ParaVT, and now let's look a bit closer at what they actually claim in this paper. The core thesis revolves around moving away from sequential tool calling to parallel tool calling in one step for long videos.
Jane: They are specifically targeting the problem where a single wrong crop error gets passed along without peer correction, and multi-turn calls just mess up the context structure of the model.
Lu: ParaVT’s approach is dispatching multiple time window crops in a single turn, which they claim leads to cleaner context and better fault tolerance compared to traditional sequential methods.
Meng: Cleaner context sounds good for accuracy, but I'm curious about how this parallel dispatch actually impacts inference cost when you have many tools being called simultaneously.
Lalam: The paper also introduces the Tool Prior Paradox, which is the central challenge they identified: pretrained tool priors that enable exploration can destabilize the cold-started structural format and expose a shortcut where skipping tools becomes rewarded under temperature sampling.
Tom: That paradox is tricky because it suggests that the very thing we use to get models to try new things—those pretraining priors—is what causes them to break their basic rules when they start learning via reinforcement learning.
Jane: To fight this, they propose PARA-GRPO, which uses Exploration Anchoring and nFrames Gating. The anchoring part fixes format issues by selectively rewarding structural tokens most prone to collapse, and the gating part creates a curriculum for training by randomizing the overview frame budget per prompt so tools are called when necessary.
Lu: The exploration anchoring mechanism is designed to stabilize the format side by using a selective reward term on those vulnerable structural token positions, and it also includes a constrained generation hook to fix only the opening reasoning tag.
Meng: That sounds like they are trying to keep the rollouts well-formed enough so they can actually be parsed into tool calls, which is crucial for any RL training pipeline.
Lalam: And then there's the nFrames Gating, which tackles the reward signal by randomizing the overview frame budget per prompt, setting up a curriculum where calling a tool yields a measurable reward signal over skipping it.
Conclusion: Tom: So, to wrap up ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning, what’s the big picture here? It’s about moving from brittle sequential chains to a parallel dispatch system for video tools.
Jane: The authors are essentially showing that just having a model that can use tools isn't enough; you need a way to make those tool uses robust and context-aware in long videos.
Lu: They propose PARA-GRPO as the solution, which combines format anchoring and reward gating to tame the trade-off between exploration and structural integrity. It shows that you can use pretrained knowledge without it breaking the model during reinforcement learning training under temperature sampling.
Meng: From a deployment standpoint, if this works, it suggests we could build more reliable agents for video understanding where we don't have to worry about cascading errors from a single bad tool execution.
Lalam: The main implication is that you can get better performance on those long video benchmarks with models like ParaVT when they are trained with PARA-GRPO, achieving a seven point nine percent average gain over the Qwen3 VL baseline.
Tom: So, in short, it’s about parallel tool use for long videos. It moves us away from the slow, error-prone sequential way and introduces a structured training method to handle the inherent instability of pretrained priors during RL training.
Zuhao Yang, Kaichen Zhang, Sudong Wang, Keming Wu, Zhongyu Yang
MiroMind · NTU · HKU-HKUST(GZ) · THU
cs.CV
Submitted: 2026-05-19
Updated: 2026-11-04
Comments: NeurIPS 2026
Code: https://github.com/EvolvingLMMs-Lab/ParaVT
Project page: https://evolvinglmms-lab.github.io/ParaVT/Abstract
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 99/100
The gist: Training large multimodal models (LMMs) via reinforcement learning (RL) to natively invoke video-processing tools has become a promising route to long-video understanding However…
Key concepts
- Tool Prior Paradox
- This paradox describes how the knowledge from pre-trained tools—which helps the model explore new actions—simultaneously destabilizes the model's learned structural format (like correct tagging) and encourages it to skip necessary tool calls during reinforcement learning training.
- Format Fragility
- This refers to when a model, while exploring tools, reverts to its initial pre-trained structure instead of following the desired output format. This manifests as incorrect tags or failing to close necessary structural boundaries in the generated text.
- Tool Necessity Gap
- This occurs when the reward signal for calling a tool is too weak compared to skipping it, especially when a direct answer is possible. The model learns to exploit this gap by skipping tools, leading to poor task accuracy despite the presence of tool-use capabilities.
Terminology
Summary
Training large multimodal models (LMMs) via reinforcement learning (RL) to natively invoke video-processing tools has become a promising route to long-video understanding However, existing native-RL methods dispatch tool calls sequentially (i. p. 0 io/ParaVT/
Abstract
Training large multimodal models (LMMs) via reinforcement learning (RL) to
natively invoke video-processing tools (e. , cropping) has become a promising
route to long-video understanding. We introduce ParaVT, the first multi-agent
end-to-end RL-trained framework for Parallel Video Tool calling, dispatching
multiple time-window crops in a single turn for cleaner context and better fault
tolerance. Yet applying standard RL to ParaVT reveals an obstacle we term the
Tool Prior Paradox: the pretrained tool priors that enable tool exploration also
destabilize cold-started structural format and expose the skip-tool reward shortcut
under temperature sampling. We introduce ParaVT, the first multi-agent
end-to-end RL-trained framework for Parallel Video Tool calling, dispatching
multiple time-window crops in a single turn for cleaner context and better fault
tolerance. Yet applying standard RL to ParaVT reveals an obstacle we term the
Tool Prior Paradox: the pretrained tool priors that enable tool exploration also
destabilize cold-started structural format and expose the skip-tool reward shortcut
under temperature sampling. Across six longvideo understanding benchmarks, ParaVT improves over the Qwen3-VL baseline
by +7. 9% on average, with PARA-GRPO lifting training-time format compliance
from 0. Across six longvideo understanding benchmarks, ParaVT improves over the Qwen3-VL baseline
by +7. 9% on average, with PARA-GRPO lifting training-time format compliance
from 0.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Pruning redundant reasoning tokens
The paper identifies the Tool Prior Paradox, which is decomposed into Format Fragility and Tool Necessity Gap Format Fragility manifests in two forms: tag-level reversion (i.e., the policy emitting the pretrained schema in place of) and structural-boundary collapse (i. p. 4 2 PARA-GRPO: Parseability-Anchored and Ratio-Gated GRPO
Format Fragility manifests in two forms: tag-level reversion (i. , the policy emitting the
pretrained <tool_code> schema in place of <tool_call>) and structural-boundary collapse (i. , the policy emitting the
pretrained <tool_code> schema in place of <tool_call>) and structural-boundary collapse (i. , failure to close </think> and </answer>). Yet applying standard RL to ParaVT reveals an obstacle we term the
Tool Prior Paradox: the pretrained tool priors that enable tool exploration also
destabilize cold-started structural format and expose the skip-tool reward shortcut
under temperature sampling. A cross-model contrast on a weaker-prior LMM
supports this claim: format stays stable but RL elicits zero tool calls, indicating that
prior strength is the shared driver of both format collapse and tool exploration.Where the paper says it
Where the paper says it
Where the paper says it
How the training signal is built
The PARA-GRPO framework introduces two complementary mechanisms to tame the paradox: Exploration Anchoring and nFrames Gating. Exploration Anchoring stabilizes the format side via two cooperating mechanisms: a selective reward term targets the few structural-token positions most vulnerable to collapse, and a Constrained Generation hook fixes only the opening reasoning tag. p. 2 We propose PARA-GRPO (Parseability-Anchored and Ratio-gAted GRPO) with Exploration
Anchoring and nFrames Gating to tame the Tool Prior Paradox (Section 3. Exploration Anchoring
stabilizes the format side via two cooperating mechanisms: a selective reward term targets the few
structural-token positions most vulnerable to collapse, and a Constrained Generation hook fixes only
the opening reasoning tag. Together they anchor rollout parseability without restricting reasoning
content or tool-call sequences. nFrames Gating tackles the reward-signal side: randomizing the
overview-frame budget per prompt creates a curriculum where a fraction of prompts cannot be
answered from overview frames alone, gating a non-trivial call/skip advantage ratio that vanilla
GRPO would otherwise average to zero. nFrames Gating tackles the reward-signal side: randomizing the
overview-frame budget per prompt creates a curriculum where a fraction of prompts cannot be
answered from overview frames alone, gating a non-trivial call/skip advantage ratio that vanilla
GRPO would otherwise average to zero. The two design choices are complementary: anchoring
keeps rollouts well-formed enough to be parseable, and only on parseable rollouts can gating credit
the tool-reward gradient.Where the paper says it
Where the paper says it
Where the paper says it
Where the gains come from
The empirical results demonstrate that ParaVT outperforms all comparable open-source 7–8B baselines on six of seven evaluation splits, with the largest gains concentrated on the long-video MCQ subset (+15.7% on LongVideoBench, +20. p. 6 Evaluation. We evaluate on six long-video benchmarks under a unified 64-frame adaptive protocol,
reporting MCQ accuracy on VideoMME [8], LongVideoBench [38], LVBench [36], MLVU [53], and
MMVU [51], and mean Intersection over Union (mIoU) on Charades-STA [11]. Evaluation. We evaluate on six long-video benchmarks under a unified 64-frame adaptive protocol,
reporting MCQ accuracy on VideoMME [8], LongVideoBench [38], LVBench [36], MLVU [53], and
MMVU [51], and mean Intersection over Union (mIoU) on Charades-STA [11]. 2% on
LVBench, +11. 5% on MLVU; an average relative gain of +7. 2 Main Results
As shown in Table 1, ParaVT outperforms all comparable open-source 7–8B baselines on six of the
seven evaluation splits. ParaVT improves on its Qwen3-VL-8B base model across every split, with the
largest gains concentrated on the long-video MCQ subset (+15. 13 while κ collapses to near zero; Exploration Anchoring (orange) lifts fτ but keeps κ
moderate; nFrames Gating (green) pushes κ off-chart while leaving fτ low; only the full PARAGRPO (blue) stabilizes both axes. PARA-GRPO resolves both extremes, reaching the highest training-time mean format reward in
Block A (fτ=0. PARA-GRPO resolves both extremes, reaching the highest training-time mean format reward in
Block A (fτ=0. 21), and strictly improves on vanilla GRPO across all six evaluation splits,
with the largest gains on LongVideoBench and MMVU.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Dispatch Mode
The parallel dispatch architecture of ParaVT re-organizes the tool-using loop as a single-turn divide-and-conquer step, dispatching K parallel invocations on disjoint temporal windows to K independent sub-agents concurrently. This yields three concrete advantages over the sequential paradigm: (i) Peer-Correctable Evidence, where a mis-localized window is outvoted by its peers rather than propagated down a serial chain; (ii) Controlled Context Growth, as returning text summaries adds only a small constant extension to the running context, while returning original frames would re-inflate it with K visual-token blocks per turn; and (iii) Bounded Inference Latency, as the K sub-agents run concurrently so the tool-using portion of the rollout is bounded by the slowest sub-agent rather than by their sum. p. 3 Returning text summaries
adds only a small constant extension to the running context, while returning original frames would
re-inflate it with K visual-token blocks per turn. (iii) Bounded Inference Latency.Where the paper says it
Conclusion
In this work, we present ParaVT, the first multi-agent end-to-end RL-trained framework that enables tool-native LMMs to dispatch Parallel Video Tool calls in a single turn for long-video reasoning, replacing brittle sequential tool chains with peer-correctable evidence aggregation while keeping inference cost flat as the number of dispatched tools grows. By identifying the central training trade-off as the Tool Prior Paradox (the dual role of pretrained tool priors in driving both tool exploration and structural-format collapse under temperature sampling), we propose PARA-GRPO, which augments standard GRPO with a parseability-anchored format reward applied only at the structural-token positions most prone to collapse, and a ratio-gated frame-budget randomization that credits tools only on prompts where they are genuinely necessary. p. 8 5 Conclusion
In this work, we present ParaVT, the first multi-agent end-to-end RL-trained framework that enables
tool-native LMMs to dispatch Parallel Video Tool calls in a single turn for long-video reasoning,
replacing brittle sequential tool chains with peer-correctable evidence aggregation while keeping
inference cost flat as the number of dispatched tools grows. By identifying the central training
trade-off as the Tool Prior Paradox (the dual role of pretrained tool priors in driving both tool
exploration and structural-format collapse under temperature sampling), we propose PARA-GRPO,
which augments standard GRPO with a parseability-anchored format reward applied only at the
structural-token positions most prone to collapse, and a ratio-gated frame-budget randomization
that credits tools only on prompts where they are genuinely necessary. 97K-sample multi-task SFT corpus and a separate 4,406-sample RL split spanning open-ended QA,
multiple-choice, and temporal grounding, ParaVT outperforms existing open-source 7–8B baselines
on six of seven long-video evaluation splits, demonstrating that anchoring format and gating tool
incentives is a transferable recipe for agentic RL as tool capabilities become increasingly internalized
in modern base LMMs. References
[1] Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao
Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al.Where the paper says it
Where the paper says it
--- Page 1 ---
ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning Zuhao Yang2,6, Kaichen Zhang3,6, Sudong Wang4, Keming Wu5,6, Zhongyu Yang2 Bo Li6 Xiaojuan Qi3 Shijian Lu2B Xingxuan Li1Lidong Bing1 1MiroMind 2NTU 3HKU 4HKUST(GZ) 5THU 6LMMs-Lab https://evolvinglmms-lab.github.io/ParaVT/
Abstract Training large multimodal models (LMMs) via reinforcement learning (RL) to natively invoke video-processing tools (e.g., cropping) has become a promising route to long-video understanding However, existing native-RL methods dispatch tool calls sequentially (i. p. 0 , cropping) has become a promising
route to long-video understanding. However, existing native-RL methods dispatch
tool calls sequentially (i.Where the paper says it
--- Page 2 ---
Greedy Decoding under Evaluation = 0; f = 1 (when correct) Closer look.crop(0:42-0:48) B well-formed Closer look. p. 1 Greedy Decoding under Evaluation
= 0; f = 1 (when correct)
<think>Closer look. </think>
<tool_call>crop(0:42-0:48)
</tool_call>
<answer> B </answer>
well-formed <think> / <tool_call> /
<answer> structure
Temperature Sampling under GRPO
= 0. </think>
<tool_code>
print(crop_video(. ))
<tool_code> in place of <tool_call>;
closing tag missing; no <answer> ×
(a) Format Fragility
0 1 2 3 4 5 6 7 8 9
RL Step
0. ))
<tool_code> in place of <tool_call>;
closing tag missing; no <answer> ×
(a) Format Fragility
0 1 2 3 4 5 6 7 8 9
RL Step
0. 25
(b) Tool Necessity Gap
Tool Calls / Sample
Task Accuracy (smoothed)
Task Accuracy (raw)
0. ))
<tool_code> in place of <tool_call>;
closing tag missing; no <answer> ×
(a) Format Fragility
0 1 2 3 4 5 6 7 8 9
RL Step
0. 25
(b) Tool Necessity Gap
Tool Calls / Sample
Task Accuracy (smoothed)
Task Accuracy (raw)
0. 25
(b) Tool Necessity Gap
Tool Calls / Sample
Task Accuracy (smoothed)
Task Accuracy (raw)
0. 8
Figure 1: Two Failure Modes of the Tool Prior Paradox. 25
(b) Tool Necessity Gap
Tool Calls / Sample
Task Accuracy (smoothed)
Task Accuracy (raw)
0. 8
Figure 1: Two Failure Modes of the Tool Prior Paradox. 8
Figure 1: Two Failure Modes of the Tool Prior Paradox. (a) Format Fragility: rollouts are wellformed under greedy decoding (sampling temperature τ=0, format reward ≈ 1); under temperature
sampling within vanilla GRPO (τ=0. 8
Figure 1: Two Failure Modes of the Tool Prior Paradox. (a) Format Fragility: rollouts are wellformed under greedy decoding (sampling temperature τ=0, format reward ≈ 1); under temperature
sampling within vanilla GRPO (τ=0. (a) Format Fragility: rollouts are wellformed under greedy decoding (sampling temperature τ=0, format reward ≈ 1); under temperature
sampling within vanilla GRPO (τ=0. 7), the policy reverts to the pretrained <tool_code> tag
in place of <tool_call>, often drops closing tags, and stops emitting <answer> altogether
(fτ≈0. (a) Format Fragility: rollouts are wellformed under greedy decoding (sampling temperature τ=0, format reward ≈ 1); under temperature
sampling within vanilla GRPO (τ=0. 7), the policy reverts to the pretrained <tool_code> tag
in place of <tool_call>, often drops closing tags, and stops emitting <answer> altogether
(fτ≈0. 7), the policy reverts to the pretrained <tool_code> tag
in place of <tool_call>, often drops closing tags, and stops emitting <answer> altogether
(fτ≈0. (b) Tool Necessity Gap: tool-call count drops to near-zero within 7 steps while task
accuracy oscillates between 0. 7), the policy reverts to the pretrained <tool_code> tag
in place of <tool_call>, often drops closing tags, and stops emitting <answer> altogether
(fτ≈0. (b) Tool Necessity Gap: tool-call count drops to near-zero within 7 steps while task
accuracy oscillates between 0. As a
result, malformed rollouts cannot be parsed into
rewardable tool calls, so the GRPO advantage
signal is computed over a corrupted trajectory population before any tool-use credit can be assigned. The second is Tool Necessity Gap (Figure 1b): when uniformly-sampled overview frames suffice
to answer many prompts directly, the reward gap between “call tool” and “skip tool” rollouts is
near-zero, so GRPO’s group-normalized advantage on the call/skip dimension is also near-zero, and
the policy converges to the canonical reward-hacking shortcut of skipping tools [29]. As a
result, malformed rollouts cannot be parsed into
rewardable tool calls, so the GRPO advantage
signal is computed over a corrupted trajectory population before any tool-use credit can be assigned. The second is Tool Necessity Gap (Figure 1b): when uniformly-sampled overview frames suffice
to answer many prompts directly, the reward gap between “call tool” and “skip tool” rollouts is
near-zero, so GRPO’s group-normalized advantage on the call/skip dimension is also near-zero, and
the policy converges to the canonical reward-hacking shortcut of skipping tools [29]. 5-VL (weak prior) Qwen3-VL (strong prior)
Figure 2: Cross-Model Evidence for the Tool
Prior Paradox under Vanilla GRPO. Qwen3-
VL-8B (stronger tool prior) explores tool use
but collapses on format, while Qwen2. 5-VL (weak prior) Qwen3-VL (strong prior)
Figure 2: Cross-Model Evidence for the Tool
Prior Paradox under Vanilla GRPO. Qwen3-
VL-8B (stronger tool prior) explores tool use
but collapses on format, while Qwen2.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
--- Page 3 ---
A natural choice for end-to-end ParaVT training is Group Relative Policy Optimization (GRPO) [12] on top of a tool-native coldstarted Qwen3-VL [1] checkpoint However, vanilla GRPO exhibits two coupled training-time failures. The first is Format Fragility (Figure 1a): the SFT-learned // format is reliable under greedy decoding but, within a few vanilla-GRPO steps under temperature sampling, the policy reverts to the pretrained schema. p. 1 However, vanilla GRPO exhibits two coupled
training-time failures. The first is Format Fragility (Figure 1a): the SFT-learned
<think>/<tool_call>/<answer> format
is reliable under greedy decoding but, within
a few vanilla-GRPO steps under temperature
sampling, the policy reverts to the pretrained
<tool_code> schema. However, vanilla GRPO exhibits two coupled
training-time failures. The first is Format Fragility (Figure 1a): the SFT-learned
<think>/<tool_call>/<answer> format
is reliable under greedy decoding but, within
a few vanilla-GRPO steps under temperature
sampling, the policy reverts to the pretrained
<tool_code> schema. However, vanilla GRPO exhibits two coupled
training-time failures. The first is Format Fragility (Figure 1a): the SFT-learned
<think>/<tool_call>/<answer> format
is reliable under greedy decoding but, within
a few vanilla-GRPO steps under temperature
sampling, the policy reverts to the pretrained
<tool_code> schema. The first is Format Fragility (Figure 1a): the SFT-learned
<think>/<tool_call>/<answer> format
is reliable under greedy decoding but, within
a few vanilla-GRPO steps under temperature
sampling, the policy reverts to the pretrained
<tool_code> schema. This is a shallow override of the SFT format reminiscent of the Superficial Alignment Hypothesis [52], compounded
by the competing pretrained tool priors: the
probability mass on tool-call continuations carried over from pretraining (before SFT) that
resurfaces under RL-time temperature. The first is Format Fragility (Figure 1a): the SFT-learned
<think>/<tool_call>/<answer> format
is reliable under greedy decoding but, within
a few vanilla-GRPO steps under temperature
sampling, the policy reverts to the pretrained
<tool_code> schema. This is a shallow override of the SFT format reminiscent of the Superficial Alignment Hypothesis [52], compounded
by the competing pretrained tool priors: the
probability mass on tool-call continuations carried over from pretraining (before SFT) that
resurfaces under RL-time temperature. The first is Format Fragility (Figure 1a): the SFT-learned
<think>/<tool_call>/<answer> format
is reliable under greedy decoding but, within
a few vanilla-GRPO steps under temperature
sampling, the policy reverts to the pretrained
<tool_code> schema. This is a shallow override of the SFT format reminiscent of the Superficial Alignment Hypothesis [52], compounded
by the competing pretrained tool priors: the
probability mass on tool-call continuations carried over from pretraining (before SFT) that
resurfaces under RL-time temperature. As a
result, malformed rollouts cannot be parsed into
rewardable tool calls, so the GRPO advantage
signal is computed over a corrupted trajectory population before any tool-use credit can be assigned. The second is Tool Necessity Gap (Figure 1b): when uniformly-sampled overview frames suffice
to answer many prompts directly, the reward gap between “call tool” and “skip tool” rollouts is
near-zero, so GRPO’s group-normalized advantage on the call/skip dimension is also near-zero, and
the policy converges to the canonical reward-hacking shortcut of skipping tools [29]. As a
result, malformed rollouts cannot be parsed into
rewardable tool calls, so the GRPO advantage
signal is computed over a corrupted trajectory population before any tool-use credit can be assigned. The second is Tool Necessity Gap (Figure 1b): when uniformly-sampled overview frames suffice
to answer many prompts directly, the reward gap between “call tool” and “skip tool” rollouts is
near-zero, so GRPO’s group-normalized advantage on the call/skip dimension is also near-zero, and
the policy converges to the canonical reward-hacking shortcut of skipping tools [29].Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
--- Page 4 ---
Figure 3 Framework Comparison.(a) Sequential Tool Calling: successive turns re-include the full context, accumulating visual-token overhead; a single mis-localized crop propagates errors with no peer to correct it, yielding an error-amplified answer. p. 3 (a) Sequential Tool Calling
Video Frames + User Query
T1: crop(0:10-1:20) +5K tokens
T2: crop(2:00-2:50) +5K tokens
T3: crop(3:30-4:30) +5K tokens
Error-Amplified Answer
visual-text ratio saturates per turn
(b) Parallel Tool Calling (Ours)
Video Frames + User Query
Main Agent: Plan & Dispatch
Sub-Agent 1
(0:10-1:20)
Sub-Agent 2
(2:00-2:50)
Sub-Agent 3
(3:30-4:30)
concurrent execution
Main Agent: Gather & Reason
Evidence-Aggregated Answer
Figure 3: Framework Comparison. (a) Sequential Tool Calling: successive turns re-include the
full context, accumulating visual-token overhead; a single mis-localized crop (✗) propagates errors
with no peer to correct, yielding an error-amplified answer. (a) Sequential Tool Calling: successive turns re-include the
full context, accumulating visual-token overhead; a single mis-localized crop (✗) propagates errors
with no peer to correct, yielding an error-amplified answer. (b) Parallel Tool Calling (Ours): one
main agent dispatches K tool calls concurrently to K independent sub-agents (shown for K=3);
mis-localized peers (✗) are outvoted by correct ones (✓), yielding an evidence-aggregated answer. Within
a single turn, the main agent emits K parallel <tool_call> invocations on disjoint temporal
windows, each dispatched to one of K independent sub-agents that share weights with the main
agent. Each sub-agent grounds only its assigned window, samples a short crop, and returns a textual
summary rather than resampled frames. Within
a single turn, the main agent emits K parallel <tool_call> invocations on disjoint temporal
windows, each dispatched to one of K independent sub-agents that share weights with the main
agent. Each sub-agent grounds only its assigned window, samples a short crop, and returns a textual
summary rather than resampled frames.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
--- Page 5 ---
In ParaVT consists of three design elements: a parallel-dispatch architecture (Section 3.1.1), a two-stage training pipeline (Section 3. p. 3 1 ParaVT: Parallel Video Tool Calling for Long-Video Understanding
ParaVT consists of three design elements: a parallel-dispatch architecture (Section 3. 1), a two-stage
training pipeline (Section 3. 1), a two-stage
training pipeline (Section 3. 2), and a self-curated multi-task dataset (Section 3. 2), and a self-curated multi-task dataset (Section 3. 1 Framework Design
A common paradigm for tool-augmented long-video understanding lets the LMM decide when and
where in the video to look more closely by issuing a crop_video(start, end) function call
that returns the requested temporal segment with densely resampled frames for further inspection
(Figure 3a). ParaVT re-organizes the same loop as a single-turn divide-and-conquer step (Figure 3b). Within
a single turn, the main agent emits K parallel <tool_call> invocations on disjoint temporal
windows, each dispatched to one of K independent sub-agents that share weights with the main
agent.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
--- Page 6 ---
This single-turn parallel dispatch yields three concrete advantages over the sequential paradigm: (i) Peer-Correctable Evidence, where the main agent receives K cross-checkable summaries grounded in disjoint windows by independent sub-agents, so a mis-localized window is outvoted by its peers rather than propagated down a serial chain; (ii) Controlled Context Growth, as returning text summaries adds only a small constant extension to the running context, while returning original frames would re-inflate it with K visual-token blocks per turn; and (iii) Bounded Inference Latency, as the K sub-agents run concurrently so the tool-using portion of the rollout is bounded by the slowest sub-agent rather than by their sum.
--- Page 7 ---
RL Step 0.05 (b) Tool Necessity Gap Tool Calls / Sample Task Accuracy (smoothed) Task Accuracy (raw) =0.0,0.2,0.4,0.6,0.8 Figure 1: Two Failure Modes of the Tool Prior Paradox.(a) Format Fragility: rollouts are wellformed under greedy decoding (sampling temperature τ=0, format reward ≈ 1); under temperature sampling within vanilla GRPO (τ=0.7), the policy reverts to the pretrained tag in place of, often drops closing tags, and stops emitting altogether (fτ≈0.1) (b) Tool Necessity Gap: tool-call count drops to near-zero within 7 steps while task accuracy oscillates between 0.45 and 0.74, as the policy converges on the shortcut of skipping tools Figure 2 Cross-Model Evidence for the Tool Prior Paradox. Qwen3-VL-8B (stronger tool prior) explores tool use but collapses on format, while Qwen2.5-VL-7B (weaker tool prior) stays format-perfect yet emits zero tool calls
--- Page 8 ---
A natural choice for end-to-end ParaVT training is Group Relative Policy Optimization (GRPO) [12] on top of a tool-native coldstarted Qwen3-VL [1] checkpoint However, vanilla GRPO exhibits two coupled training-time failures. The first is Format Fragility (Figure 1a): the SFT-learned // format is reliable under greedy decoding but, within a few vanilla-GRPO steps under temperature sampling, the policy reverts to the pretrained schema. This is a shallow override of the SFT format reminiscent of the Superficial Alignment Hypothesis [52], compounded by the competing pretrained tool priors: the probability mass on tool-call continuations carried over from pretraining (before SFT) that resurfaces under RL-time temperature. As a result, malformed rollouts cannot be parsed into rewardable tool calls, so the GRPO advantage signal is computed over a corrupted trajectory population before any tool-use credit can be assigned The second is Tool Necessity Gap (Figure 1b): when uniformly-sampled overview frames suffice to answer many prompts directly, the reward gap between “call tool” and “skip tool” rollouts is near-zero, so GRPO’s group-normalized advantage on the call/skip dimension is also near-zero, and the policy converges to the canonical reward-hacking shortcut of skipping tools To probe the role of pretrained tool priors, we replicate the same setup on Qwen2.5-VL [25] (with much weaker tool priors than Qwen3-VL) under identical hyperparameters (Figure 2): its format stays near-perfect yet RL elicits no tool calls
--- Page 9 ---
To probe the role of pretrained tool priors, we replicate the same setup on Qwen2.5-VL [25] (with much weaker tool priors than Qwen3-VL) under identical hyperparameters (Figure 2): its format stays near-perfect yet RL elicits no tool calls A natural choice for end-to-end ParaVT training is Group Relative Policy Optimization (GRPO) [12] on top of a tool-native coldstarted Qwen3-VL [1] checkpoint However, vanilla GRPO exhibits two coupled training-time failures. p. 1 A natural choice for end-to-end ParaVT training is Group Relative Policy Optimization
(GRPO) [12] on top of a tool-native coldstarted Qwen3-VL [1] checkpoint. However, vanilla GRPO exhibits two coupled
training-time failures. As a
result, malformed rollouts cannot be parsed into
rewardable tool calls, so the GRPO advantage
signal is computed over a corrupted trajectory population before any tool-use credit can be assigned. The second is Tool Necessity Gap (Figure 1b): when uniformly-sampled overview frames suffice
to answer many prompts directly, the reward gap between “call tool” and “skip tool” rollouts is
near-zero, so GRPO’s group-normalized advantage on the call/skip dimension is also near-zero, and
the policy converges to the canonical reward-hacking shortcut of skipping tools [29].Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Parallel Video Tool Calling via ParaVT enables a main agent to issue multiple time-window crops in a single turn for cleaner context and better fault tolerance, replacing brittle sequential tool chains with peer-correctable evidence aggregation.
-
PARA-GRPO (Parseability-Anchored and Ratio-gated GRPO) augments standard RL with two complementary mechanisms: (i) a targeted format reward applied only at the structural-token positions most prone to collapse, and (ii) a per-prompt frame-budget randomization that creates training prompts where calling the tool yields a measurable reward signal over skipping it.
-
Exploration Anchoring repairs rollout parseability at the structural-token boundaries where collapse concentrates by using a selective reward term that targets specific token positions, while nFrames Gating creates the missing gap by randomizing the overview-frame budget to generate a non-trivial reward contrast between tool-calling and tool-skipping rollouts.
-
The improved system can achieve better performance on long video understanding benchmarks, such as LongVideoBench (60.4) and MMVU (68.6), demonstrating that anchoring format and gating tool incentives is a
transferable recipe for agentic RL.
Abstract
Training large multimodal models (LMMs) via reinforcement learning (RL) to natively invoke video-processing tools (e.g., cropping) has become a promising route to long-video understanding. However, most existing agentic Video-LMMs dispatch tool calls sequentially (i.e., one per turn). Sequential tool calling can propagate localization errors, accumulate irrelevant context, and require an additional main-model generation for each successive crop. We introduce ParaVT, a multi-agent RL framework for Parallel Video Tool calling, dispatching multiple time-window crops in a single turn, allowing evidence from multiple temporal windows to be examined concurrently. Yet applying standard Group Relative Policy Optimization (GRPO) to ParaVT reveals an obstacle we term the Tool Prior Paradox: inherited tool-use priors appear to support exploration while competing with the output format learned during supervised fine-tuning. Standard RL can also settle on tool skipping. A comparison with a model with weaker tool-use priors shows stable formatting but no tool calls during RL, consistent with a trade-off between format stability and tool exploration. We propose PARA-GRPO (Parseability-Anchored and Ratio-gAted GRPO), which augments standard RL with two complementary mechanisms: (i) a format reward that encourages tag closure and penalizes unclosed reasoning blocks, and (ii) a per-prompt randomization of the overview-frame budget to create a reward advantage for tool use when the overview provides insufficient evidence. Across six long-video understanding benchmarks, ParaVT improves over the Qwen3-VL baseline by +7.9% on average, with PARA-GRPO raising training-time format reward from 0.13 to a peak of 0.64. These findings highlight the need to account for inherited tool-use priors when designing RL methods for long-video understanding.
Sources
- Qwen3-VL Technical Report
- VideoChat-M1: Collaborative Policy Planning for Video Understanding via Multi-Agent Reinforcement Learning
- Scaling RL to Long Videos
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoning
- Video-R1: Reinforcing Video Reasoning in MLLMs
- LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoning
- AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models
- GPT-4o System Card
- SAGE: Training Smart Any-Horizon Agents for Long Video Reasoning with Reinforcement Learning
- VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning
- LongVideoAgent: Multi-Agent Reasoning with Long Videos
- MUSEG: Reinforcing Video Temporal Understanding via Timestamp-Aware Multi-Segment Grounding
- MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning
- Sparse but Critical: A Token-Level Analysis of Distributional Shifts in RLVR Fine-Tuning of LLMs
- Conan: Progressive Learning to Reason Like a Detective over Multi-Scale Visual Evidence
- LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL
- Safety Alignment Should Be Made More Than Just a Few Tokens Deep
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs