Video-FLAIR: Not Whether to Reason, But How
summary
The gist
Multimodal LLMs struggle to adapt their reasoning to each query because existing methods apply a uniform reasoning strategy across queries…
In short
Video-FLAIR addresses how multimodal LLMs should reason by introducing a framework that learns to select an appropriate reasoning mode for each query using reinforcement learning. It defines three modes: PERCEPT, COMPOSE, and DELIBERATE. By training on comparative rewards across these modes for the same prompt, the model learns an adaptive strategy instead of applying a fixed one.
Key concepts
- PERCEPT
- This reasoning mode involves mapping evidence directly to an answer. It is suitable for straightforward queries where a direct extraction from the input data is sufficient. The model focuses on finding the most relevant information in the provided video or text and generating a concise response based solely on that evidence.
- COMPOSE
- The COMPOSE mode builds answers through a sequence of observations. Instead of jumping straight to an answer, this mode involves synthesizing information by iteratively processing observations. It is useful for queries requiring a step-by-step construction or narrative flow from the available evidence.
- DELIBERATE
- The DELIBERATE mode is used when deep reasoning about competing hypotheses is necessary. In this mode, the model revisits the same evidence multiple times to explore different interpretations and weigh conflicting information. This allows for more complex problem-solving where careful consideration of all aspects of the evidence is required.
Terminology used across episodes
This episode discusses
- Video-FLAIR: Not Whether to Reason, But How · Paper Radio
- Qwen3-VL Technical Report
- Qwen2.5-VL Technical Report
- Perception Before Reasoning: Two-Stage Reinforcement Learning for Visual Reasoning in Vision-Language Models
- GraphThinker: Reinforcing Temporally Grounded Video Reasoning with Event Graph Thinking
- What MLLMs Learn about When they Learn about Multimodal Reasoning · Paper Radio
- OverThink: Slowdown Attacks on Reasoning LLMs · Paper Radio
- Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models
- AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforcement Learning
- Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMs
- Video-R2: Reinforcing Consistent and Grounded Reasoning in Multimodal Language Models
- Video-CoM: Interactive Video Reasoning via Chain of Manipulations
- Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMs
- Omni-AutoThink: Adaptive Multimodal Reasoning via Reinforcement Learning
- MM-Vet v2: A Challenging Benchmark to Evaluate Large Multimodal Models for Integrated Capabilities
- KAT-V1: Kwai-AutoThink Technical Report
- Rethinking Chain-of-Thought Reasoning for Videos
The paper
Video-FLAIR: Not Whether to Reason, But How · Read on arXiv
Arizona State University
Multimodal queries can require different types of reasoning. Some can be answered via perceptual reasoning, extracting information directly from the visual signal, while others require compositional reasoning that combines observations or deliberative reasoning that evaluates competing hypotheses. However, many existing methods apply a uniform reasoning strategy across queries, leading to unnecessary computation on simple tasks and insufficient reasoning on complex ones. We introduce Video-FLAIR, a training framework that learns to select the appropriate reasoning mode for each query using reinforcement learning. During training, the model generates responses under all three modes for the same prompt, enabling direct comparison. A composite reward compares these responses to favor the most effective one based on correctness, grounding, and cost, while discouraging unsupported or misaligned deliberation. This yields a supervision signal for learning adaptive reasoning without per-query annotations. Video-FLAIR improves accuracy over the Qwen2.5-VL base model by +5.4 on MathVista, +4.8 on Video-Holmes, and +4.8 on Video-MMMU, while reducing average token usage to 95 compared to 417 for always-thinking baselines.
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "Video-FLAIR: Not Whether to Reason, But How".
Jane: Here's the gist of the paper. Multimodal LLMs struggle to adapt their reasoning to each query because existing methods apply a uniform reasoning strategy across queries, leading to unnecessary computation on simple tasks and insufficient reasoning on complex ones Video-FLAIR introduces a training framework that learns to select the appropriate reasoning mode for each query using reinforcement learning.
Tom: Defining Reasoning Modes The paper defines three distinct reasoning modes: PERCEPT, COMPOSE, and DELIBERATE PERCEPT maps directly from evidence to answer, COMPOSE builds answers through a concise sequence of observations, and DELIBERATE revisits the same evidence to reason deeply about competing hypotheses.
Jane: How the training signal is built Video-FLAIR generates responses under all three modes for the same prompt during training, enabling direct comparison A composite reward compares these responses to favor the most effective one based on correctness, grounding, and cost This yields a supervision signal for learning adaptive reasoning without per-query annotations.
Tom: The reward function includes components like Rans for correctness and Rcost for cost efficiency Where the gains come from Video-FLAIR improves accuracy over the Qwen2.5-VL base model by +5.4 on MathVista, +4.8 on Video-Holmes, and +4.8 on Video-MMMU while reducing average token usage to 95 compared to 417 for always-thinking baselines.
Title and authors: Tom: So we're looking at this paper called "Video-FLAIR: Not Whether to Reason, But How," and the idea here is that current multimodal models just use one fixed reasoning strategy for every single query, and that causes problems—either wasting computation on easy stuff or totally missing the mark on hard questions.
Jane: That sounds like a real headache for anyone trying to get good results from these large vision models. What does this paper actually propose to fix that uniform approach?
Tom: They introduce a training framework using reinforcement learning that teaches the model how to pick the right reasoning mode for each query based on how complex it is.
Lu: It defines three specific modes: PERCEPT, COMPOSE, and DELIBERATE. PERCEPT is about just pulling information directly from the visual signal, like answering "What sporting event is being broadcasted?"
Meng: And COMPOSE builds answers by stringing together a short sequence of observations from the video evidence. That sounds like it’s for summarizing stuff, you know?
Jane: Right, and DELIBERATE is for when the query needs deep thinking; it involves looking back at the same video evidence multiple times to evaluate different hypotheses.
Tom: The training setup is what's really interesting here: they generate responses under all three modes for the very same prompt during training so they can directly compare them.
Lu: They create a composite reward system that scores these responses based on correctness, how well they are grounded in the video, and how much computation it costs.
Jane: So instead of having to manually label every query with which mode is best, the model learns by comparing its own outputs against this composite reward signal.
Tom: That way they get a supervision signal for learning adaptive reasoning without needing those tedious per-query annotations. They even include things like Rans for correctness and Rcost for cost efficiency in that reward function.
Meng: I'm wondering about the cost part, because when you deploy these things, token usage is everything. The paper mentions a big jump in efficiency compared to always-thinking baselines on simple tasks.
Jane: It sounds like the framework learns how to select modes through structured rollouts that compare different strategies for the same prompt, which allows the model to learn adaptivity instead of just using a fixed strategy.
Tom: And they show some solid gains too; they improved accuracy over models like Qwen2 point 5-VL on MathVista by five point four percent and on Video-Holmes by four point eight percent <ref:2608.26495#pg1>.
Lu: But what really stands out is the token usage reduction, dropping to ninety-five compared to a baseline of four hundred seventeen tokens for always thinking, which suggests it’s actually more efficient overall.
Jane: So if I'm listening to this paper, the main thing I should take away is that we can move beyond a one-size-fits-all approach and let the model choose its thinking style.
Title and authors: Tom: Exactly. And looking at the ablation studies, they confirm that each reward component actually contributes independently to updating the policy, which means you can see exactly what part of the feedback is driving which kind of learning.
Meng: I'm curious about how this affects real deployment—does this adaptive selection translate well when a query is really ambiguous?
Jane: The paper suggests that this system can perform better on complex queries because the cost penalties are stronger for simple queries, which discourages unnecessary deliberation, but they remain weaker for complex ones so it allows deeper reasoning when it’s truly needed.
Lu: And the component breakdown showed that removing the selection supervision actually caused accuracy to drop while token usage went up, proving that Rselect is really doing the heavy lifting for the adaptive aspect.
Tom: It shows that without learning how to select modes, you just end up with a less efficient system anyway. The verifier reward, Rverifier, also provides dense grounding feedback which boosts reasoning quality even if it doesn't always push the benchmark accuracy up on every single test.
Meng: So what are the limitations? Where does this approach fall short when we actually run it at scale?
Lu: The authors mention that while ARM2 expands the action space to multiple reasoning formats, it still relies on fixed SFT defined templates rather than truly learning how to select among them effectively.
Jane: That means even with this reinforcement learning setup, there's still a constraint on what reasoning formats the model can actually choose from initially.
Tom: It’s not that it solves everything, but it provides a much cleaner way to structure the RL training for multimodal reasoning compared to just applying fixed SFT templates.
Meng: For production use, I need to know if this complexity adds too much overhead on the inference side when we have millions of queries coming in.
Jane: That’s the practical concern, isn't it? But for now, what this paper shows is a solid way to make the AI system smarter about *how* it thinks instead of just hoping it guesses correctly.
Tom: So to wrap up on "Video-FLAIR: Not Whether to Reason, But How," this paper is about moving from applying a single reasoning strategy everywhere to training models that can dynamically select between perceptual, compositional, and deliberative modes based on the query needs.
Lu: It shows how you can use a composite reward signal built from correctness, grounding, and cost to supervise that selection process.
Meng: It’s an interesting framework for managing complexity in multimodal inputs without requiring us to manually annotate every single tricky question we face.
Jane: It really gives us a direction on how to build systems that are smarter about their own reasoning process rather than just bigger models that try to do everything at once.
The paper's summary: Tom: So we're looking at this paper called "Video-FLAIR: Not Whether to Reason, But How." It’s about fixing that problem where multimodal models just use one fixed way to think for every single question they get.
Jane: Right, so what did the authors actually propose? They basically suggest we stop forcing the model into a single mode—like just saying "PERCEPT" all the time—and instead let it decide which thinking style fits the situation.
Tom: Exactly. They define three distinct modes for reasoning: PERCEPT, COMPOSE, and DELIBERATE. PERCEPT is just pulling information straight from the video evidence to answer something simple.
Lu: COMPOSE builds answers by creating a short sequence of observations drawn from that video evidence, which is good for summarizing stuff up. And DELIBERATE means it looks back at the same evidence over and over to really think deeply about different possibilities or hypotheses.
Jane: That makes sense for a listener who isn't reading code. So, if I ask a simple question, it uses PERCEPT. If I ask something complex that needs deep thought, it switches to DELIBERATE.
Tom: The training method is clever because they generate responses under all three modes for the exact same prompt during training so they can compare them directly against each other.
Lu: And they use a composite reward system to score those responses based on correctness, how well the answer is grounded in the video, and how much computational cost it takes.
Jane: That means instead of us manually labeling every query as "this one needs deliberation," the AI learns by comparing its own different reasoning strategies against this combined score.
Tom: It gives them a supervision signal without needing those tedious per-query annotations that used to take forever to collect. They are using things like Rans for correctness and Rcost for efficiency in that reward function.
Meng: I'm wondering about the cost part, because when you deploy these things, token usage is everything. The paper mentions a big jump in efficiency compared to always-thinking baselines on simple tasks.
Jane: It sounds like the framework learns how to select modes through structured rollouts that compare different strategies for the same prompt, which allows the model to learn adaptivity instead of just using a fixed strategy.
Tom: They show some solid gains too; they improved accuracy over models like Qwen2 point five four percent on MathVista and four point eight percent on Video-Holmes. But what really stands out is the token usage reduction, dropping to ninety-five compared to four hundred seventeen tokens for always thinking.
Lu: So if I'm listening to this paper, the main thing I should take away is that we can move beyond a one-size-fits-all approach and let the model choose its thinking style.
Jane: Exactly. And looking at the ablation studies, they confirm that each reward component actually contributes independently to updating the policy, which means you can see exactly what part of the feedback is driving which kind of learning.
Tom: It shows that without learning how to select modes, you just end up with a less efficient system anyway. The verifier reward also provides dense grounding feedback which boosts reasoning quality even if it doesn't always push the benchmark accuracy up on every single test.
Jane: So this isn't just about making the model bigger or more powerful; it's about teaching it better self-awareness about what kind of thinking is required for a given task.
Tom: That adaptive selection capability means we can finally build systems that are smarter about their own reasoning process rather than just hoping they guess correctly.
Jane: And that opens up new avenues for complex multimodal tasks where the complexity changes from one moment to the next, like in analyzing long video feeds or intricate scientific diagrams.
The paper's improvements: Tom: So we're looking at how they actually improved things in this framework called Video-FLAIR and what that means for real deployment. It wasn't just about getting a baseline score higher, though they did that on MathVista and Video-Holmes.
Jane: What’s the big win here? The paper suggests several tweaks to the reward system and how it handles information within the model itself.
Lu: One major thing they focused on was token-level credit shaping. This is like giving different weights to different pieces of evidence in the video, like those specific spans with tags or key objects.
Tom: So instead of giving every word in the response equal importance, they concentrate the gradient updates on the most crucial information, which is a clean way to guide what gets learned.
Jane: That sounds smart for making sure the model focuses its thinking energy where it matters most in a long video sequence. And they also changed how cost penalties work based on query complexity.
Tom: They made the cost penalties stronger for simple queries, which discourages unnecessary deliberation there, but they keep them weaker for complex ones so it allows deeper reasoning when needed.
Meng: That's interesting from an engineering standpoint because it directly controls the trade-off between speed and quality based on the task. But what about what happens if we try to remove that mode selection supervision?
Lu: The ablation studies showed that removing that selection supervision actually caused accuracy to drop while token usage went up, proving that learning how to select modes is essential for efficiency.
Jane: That means if you just let it guess the mode randomly, it doesn't help the system; it actively needs the reinforcement learning to learn when to switch gears.
Tom: And they showed that even a component like the verifier reward, which gives dense grounding feedback, contributes independently to updates and helps reasoning quality without necessarily boosting benchmark accuracy on every single test.
Jane: So what this changes for us is that we’re moving toward systems that aren't just trained to follow instructions; they’re trained to understand the *type* of instruction they are receiving and then adjusting their internal thinking process accordingly.
Lu: It shifts the focus from just scaling up the model size to scaling up its adaptive intelligence, which is a bigger problem for complex reasoning tasks.
Tom: And if we look at it in terms of what that means for deployment, it’s about making sure the AI doesn't waste computation on easy things and doesn't run away with complexity when it needs to be simple.
Jane: It makes the system much more robust across a wider range of inputs, which is exactly what we need for real world applications where queries are never perfectly uniform.
Conclusion: Tom: So to wrap up, Video-FLAIR is about moving away from a fixed reasoning strategy for multimodal models and instead training them to pick the right thinking mode based on what’s in the query.
Jane: Exactly, it’s about teaching these AI systems how to adapt their internal approach—choosing between perceiving, composing, or deliberating—instead of just sticking to one method.
Lu: The authors showed that this adaptive reasoning selection actually improves performance on complex tasks because the model gets better at choosing the right tool for the job.
Meng: From a deployment standpoint, it’s a nice way to control inference cost by making simple queries cheap and complex ones more expensive in terms of computation.
Lalam: I’m pretty sure this is a solid direction because it addresses the core problem of uniform reasoning that plagues most current multimodal setups.
Tom: That adaptive selection capability means we can finally build systems that are smarter about their own reasoning process rather than just hoping they guess correctly on every single input.
Jane: It opens up new avenues for complex tasks where the query's nature changes constantly, like analyzing long videos or intricate scientific diagrams that require different types of deep thought.
Lu: I think the mechanism with token-level credit shaping is a really elegant way to implement this without needing massive amounts of extra training data for mode selection.
Meng: It’s elegant, but I still wonder about the inference overhead when you have millions of queries hitting that decision point every second.
Tom: We’ll have to see how the actual runtime scales, but if it manages to reduce token usage while maintaining accuracy gains on benchmarks like MathVista, that’s a win for production.
Jane: It really shows us that we need to stop thinking of these models as single entities and start thinking about them as flexible reasoning engines.
Lalam: I’m pretty sure this is a solid direction because it addresses the core problem of uniform reasoning that plagues most current multimodal setups.
More episodes
- 2603.28595-Optimistic Actor-Critic with Parametric Policies for Linear Markov Decision Processes
- 2603.29765-Training-Free Dynamic Upcycling of Expert Language Models
- 2605.00778-Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint
- 2605.06906-TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
- 2605.09165-Sparse Layers are Critical to Scaling Looped Language Models
- 2605.09439-Inverse Design for Conditional Distribution Matching
- 2605.10313-Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
- 2605.10878-Minimum Neural Weight Norm = Kolmogorov Complexity
- 2605.11617-MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound
- 2605.15085-From Data to Action: Accelerating Refinery Optimization with AI