Thinking with Images as Continuous Actions: Numerical Visual Chain-of-Thought

arXiv:2602.23959 · cs.CV · Submitted 2026-02-27 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "Thinking with Images as Continuous Actions".

Jane: The gist The Numerical Visual Chain-of-Thought (NV-CoT) framework enables Multimodal Large Language Models (MLLMs) to reason over images using continuous numerical coordinates instead of discrete tokens,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we've covered the high level idea of NV-CoT, focusing on how it moves from discrete tokens to continuous numerical coordinates for region grounding. The core claim is that this approach overcomes the limitations of textified coordinates, which cause semantic fragmentation, and fixed patches, which limit selection.

Jane: Exactly. The thesis is that by expanding the MLLM action space into a continuous Euclidean space and allowing the model to directly generate bounding-box coordinates as actions with minimal architectural change, we get more flexible and precise localization in the visual world >

Lu: What they claim is that this expansion allows models to reason over images using these continuous numerical coordinates, which is fundamentally different from previous methods like text-based approaches that use tokens like "four eight" or patch-based approaches constrained by fixed spatial granularity > <ref:2602.23959#pg2>

Meng: So, the key difference seems to be that NV-CoT allows for flexible and precise localization because it’s predicting continuous values instead of discrete text tokens, which is a big deal for accuracy.

Lalam: And they support this with two training paradigms: supervised fine-tuning and reinforcement learning, showing it works across different optimization settings >

Tom: They detail how they implement this in SFT by using a regression loss to supervise the continuous bounding-box coordinates against the ground truth, defining it as LSFT(theta) equals X t∈TV minus log pi theta(ot o<t, q, I0) plus lambda X t∈TA mu t minus b* t squared >

Jane: That regression loss is crucial because it directly supervises the continuous nature of the output, contrasting with standard cross-entropy losses that ignore geometric proximity >

Lu: For reinforcement learning, they adapt it by replacing categorical policies with a Gaussian policy over coordinates and introducing stochasticity via reparameterized sampling to model this continuous coordinate regression >

Meng: So in RL, they are using a Gaussian policy where the model predicts both the mean and the standard deviation of the bounding box distribution and then samples from that distribution for its actions >

Lalam: And they use analytic importance ratios based on Gaussian likelihoods for GRPO-style RL to compute these importance ratios, which makes continuous localization compatible with mainstream reinforcement learning algorithms >

Tom: They also introduce a Laplace policy variant, using the Laplace distribution and an L1 loss in supervised fine-tuning to encourage sharper and more robust localization predictions >

Jane: So it's not just one method; they are exploring different ways to model this continuous action space, which includes Gaussian policies, Laplace policies, and specific loss functions for both SFT and RL >

Lu: The results on three benchmarks against eight baseline models demonstrate that NV-CoT significantly improves localization accuracy, final answer accuracy, and convergence speed across the board >

Meng: That's a solid set of results. It shows that this continuous action approach is indeed better than the methods they compared against in terms of how accurate the localization is and how fast it learns >

Lalam: And qualitatively, visualizations show NV-CoT consistently generates tighter bounding boxes compared to the backbone model, proving that modeling coordinates as continuous actions captures spatial structure more effectively >

Conclusion: Tom: So looking at "Thinking with Images as Continuous Actions: Numerical Visual Chain-of-Thought," it seems like the big move here is ditching the text tokens for continuous numerical coordinates for region grounding >

Jane: It really is. The authors are showing that this numerical approach, whether trained with supervised fine-tuning or reinforced with reinforcement learning, leads to better localization precision and final answer accuracy >

Lu: What this means in practice is that we can build MLLMs that are much better at tasks requiring precise spatial reasoning over images than the current state-of-the-art methods allow >

Meng: For deployment, it implies we can move towards systems where region grounding is inherently more robust because they aren't relying on brittle text representations or fixed grid structures >

Lalam: It’s a plug-and-play framework that mitigates the modality mismatch and semantic fragmentation issues inherent in text-based visual chain-of-thought methods while also avoiding the limitations of fixed patch designs >

Tom: So, it's not just a tweak; it's a fundamental change in how we teach these models to interact with images spatially >

Jane: And they’re showing that iterative refinement, like allowing multiple zoom-in tool calls, actually helps the model perform better when thinking with images >

Lu: So the implication is that for visual question answering or optical character recognition, we can expect much more accurate and efficient region grounding because the model learns to reason in this continuous numerical space >

Meng: It points towards a future where vision models can be trained to be inherently better at spatial tasks if we give them the right continuous action space to explore >

Lalam: This work is about advancing multimodal large language models in visual reasoning, particularly through more accurate and efficient region grounding >

Nanyang Technological University · Chalmers University of Technology and University of Gothenburg · University of Science and Technology of China

cs.CV

Submitted: 2026-02-27

Updated: 2026-10-15

Comments: Accepted at NeurIPS 2026, Oral (Top 0.4%)

Code: https://github.com/kesenzhao/NV-CoT

Project page: https://kesenzhao.github.io/NV-CoT

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 85/100

The gist: The gist The Numerical Visual Chain-of-Thought (NV-CoT) framework enables Multimodal Large Language Models (MLLMs) to reason over images using continuous numerical coordinates instead of discrete

Key concepts

Continuous Action Space
Instead of choosing from a fixed list of text tokens, NV-CoT treats the output as a continuous set of numerical values representing coordinates (like [x1, y1, x2, y2]). This allows the model to select any precise location within an image space. This contrasts with old methods that used discrete patches or textified coordinates.
Gaussian/Laplace Policy
For reinforcement learning, NV-CoT replaces categorical policies with a Gaussian policy over the continuous coordinate space. This policy models both the mean and standard deviation of the bounding box distribution. Stochasticity is introduced using reparameterized sampling to allow for varied, precise spatial predictions.
Regression Loss (SFT)
During supervised fine-tuning, NV-CoT uses a regression loss instead of token cross-entropy. This loss supervises the model to predict the actual continuous bounding box coordinates by minimizing the difference between its predicted mean and the ground truth bounding box.

Terminology

Summary

The gist The Numerical Visual Chain-of-Thought (NV-CoT) framework enables Multimodal Large Language Models (MLLMs) to reason over images using continuous numerical coordinates instead of discrete tokens, significantly improving localization precision and final answer accuracy across supervised fine-tuning and reinforcement learning settings.

How it works

  1. NV-CoT expands the MLLM action space from discrete vocabulary tokens to a continuous Euclidean space, allowing models to directly generate bounding-box coordinates as actions with only minimal architectural modification. This is achieved by extending the standard LLM head with four coordinate outputs that directly predict the numerical bounding box coordinates (e.g., [x1, y1, x2, y2]).

How it works

  1. In the supervised fine-tuning (SFT) stage, NV-CoT replaces token-level cross-entropy with a regression loss to supervise continuous bounding-box coordinates. The loss function is defined as LSFT(θ) = X t∈TV - log πθ(ot o<t, q, I0) + λ X t∈TA µt − b∗ t squared, where µ is the mean and b∗ is the ground-truth bounding box.

How it works

  1. For reinforcement learning (RL), NV-CoT extends to a stochastic Gaussian policy over the action space A, which requires replacing categorical policies with a Gaussian policy that explicitly models continuous coordinate regression. Stochasticity is introduced via the reparameterization trick, where the model predicts both the mean and standard deviation of the bounding box distribution and samples coordinates accordingly.

How it works

  1. The importance ratio for continuous actions in GRPO-style RL is computed using Gaussian likelihoods based on squared Mahalanobis distances. The objective function optimized is LGRPO(θ) = Eq,I0, G i=1∼πθold h 1 GT X G i=1 X T t=1 min r(i) t(θ)A(i) t, clipr(i) t(θ), 1 − ε, 1 + ε A(i) t - βKL(πθ∥πref) i.

How it works

  1. NV-CoT is extended with a Laplace policy variant, motivated by the empirical robustness of l1-type losses for localization. This involves modeling the coordinate action as b ∼ Laplace(µ, α), and in SFT, this replaces the l2 loss with an l1 loss to encourage sharper and more robust localization.

How it works

  1. Extensive experiments on three benchmarks against eight representative visual reasoning baselines demonstrate that NV-CoT significantly improves localization precision and final answer accuracy, while also accelerating training convergence. The results show that NV-CoT consistently improves localization accuracy, final answer accuracy, and convergence speed across both SFT and RL.

How it works

  1. Qualitative visualizations demonstrate that NV-CoT (red) consistently generates tighter and more accurate bounding boxes compared to the backbone model (blue), showing improved localization capability. The results show that modeling bounding-box coordinates as continuous actions enables NV-CoT to capture spatial structure more effectively, resulting in improved localization quality and reduced ambiguity in region grounding.

How it works

  1. The paper compares NV-CoT against eight state-of-the-art MLLM baselines, including open-source models, supervised fine-tuning based visual CoT methods, and reinforcement learning based visual CoT methods. NV-CoT consistently outperforms the backbone models across all benchmarks under both SFT and RL.

How it works

  1. The study concludes that replacing text-based bounding-box prediction with the numerical one benefits both SFT and RL training paradigms. Furthermore, allowing multiple zoom-in tool calls further improves “thinking with images” performance, indicating that iterative region selection and refinement is beneficial for fine-grained visual reasoning.

How it works

  1. The work presents a plug-and-play framework that enables MLLMs to directly generate continuous boundingbox coordinates instead of textual tokens, mitigating the modality mismatch and semantic fragmentation of textbased visual CoT while avoiding the fixed-granularity limitation of patch-based designs. This approach improves localization precision and training efficiency, which may benefit a wide range of downstream applications such as visual question answering and optical character recognition.

How it works

  1. The research relies on standard benchmark datasets commonly used in the research community, meaning that the work does not introduce new data sources, deployable systems, or application-specific pipelines. This paper presents work whose primary goal is to advance the capabilities of multimodal large language models in visual reasoning, particularly through more accurate and efficient region grounding. The impact statements note that this work does not introduce novel ethical risks beyond those already well recognized in the field of MLLMs. The authors state that ChatGPT was employed solely for minor editorial assistance, such as improving grammar and readability. This paper presents work whose primary goal is to advance the capabilities of multimodal large language models in visual reasoning, particularly through more accurate and efficient region grounding. The impact statements note that this work does not introduce new data sources, deployable systems, or application-specific pipelines. The authors state that ChatGPT was employed solely for minor editorial assistance, such as improving grammar and readability. This paper presents work whose primary goal is to advance the capabilities of multimodal large language models in visual reasoning, particularly through more accurate and efficient region grounding. The impact statements note that this work does not introduce new data sources, deployable systems, or application-specific pipelines. The authors state that ChatGPT was employed solely for minor editorial assistance, such as improving grammar and readability. This paper presents work whose primary goal is to advance the capabilities of multimodal large language models in visual reasoning, particularly through more accurate and efficient region grounding. The impact statements note that this work does not introduce new data sources, deployable systems, or application-specific pipelines. The authors state that ChatGPT was employed solely for minor editorial assistance, such as improving grammar and readability. This paper presents work whose primary goal is to advance the capabilities of multimodal large language models in visual reasoning, particularly through more accurate and efficient region grounding. The impact statements note that this work does not introduce new data sources, deployable systems, or application-specific pipelines. The authors state that ChatGPT was employed solely for minor editorial assistance, such as improving grammar and readability. This paper presents work whose primary goal is to advance the capabilities of multimodal large language models in visual reasoning, particularly through more accurate and efficient region grounding. The impact statements note that this work does not introduce new data sources, deployable systems, or application-specific pipelines. The authors state that ChatGPT was employed solely for minor editorial assistance, such as improving grammar and readability. This paper presents work whose primary goal is to advance the capabilities of multimodal large language models in visual reasoning, particularly through more accurate and efficient region grounding. The impact statements note that this work does not introduce new data sources, deployable systems, or application-specific pipelines. The authors state that ChatGPT was employed solely for minor editorial assistance, such as improving grammar and readability. This paper presents work whose primary goal is to advance the capabilities of multimodal large language models in visual reasoning, particularly through more accurate and efficient region grounding. The impact statements note that this work does not introduce new data sources, deployable systems, or application-specific pipelines. The authors state that ChatGPT was employed solely for minor editorial assistance, such as improving grammar and readability. This paper presents work whose primary goal is to advance the capabilities of multimodal large language models in visual reasoning, particularly through more accurate and efficient region grounding. The impact statements note that this work does not introduce new data sources, deployable systems, or application-specific pipelines. The authors state that ChatGPT was employed solely for minor editorial assistance, such as improving grammar and readability. This paper presents work whose primary goal is to advance the capabilities of multimodal large language models in visual reasoning, particularly through more accurate and efficient region grounding. The impact statements note that this work does not introduce new data sources, deployable systems, or application-specific pipelines. The authors state that ChatGPT was employed solely for minor editorial assistance, such as improving grammar and readability. This paper presents work whose primary goal is to advance the capabilities of multimodal large language models in visual reasoning, particularly through more accurate and efficient region grounding. The impact statements note that this work does not introduce new data sources, deployable systems, or application-specific pipelines.

Improvements for AI systems

  1. Bold action space expansion: Expand MLLM action space from discrete vocabulary tokens to a continuous Euclidean space by introducing four coordinate heads to directly predict numerical bounding box coordinates [x1, y1, x2, y2]. This enables models to directly generate numerical bounding box coordinates as actions with only minimal architectural modification.

  2. Supervised fine-tuning enhancement: Replace the standard token-level cross-entropy loss with a regression loss to supervise continuous bounding-box coordinates using an objective like LSFT(θ) = Xt∈TV - log πθ(ot o) + λXt∈TA µt − b∗it ∥22 to supervise continuous localization during SFT.

  3. Reinforcement learning compatibility: Implement a Gaussian policy with reparameterized sampling where b(i) = µ + σϵ(i), ε(i) ∼ N (0, I) to enable low-variance gradient estimation in GRPO-style policy optimization, making continuous localization compatible with mainstream RL algorithms.

  4. Robust localization objective: Adopt the Laplace policy variant for RL, modeling the coordinate action as b ∼ Laplace(µ, α), whose probability density function is p(b µ, α) = 1/(2α) exp − b − µ1/α, which results in an SFT objective that replaces the L2 loss with an l1 loss to encourage sharper and more robust localization.

  5. Improved reasoning fidelity: Use the derived importance ratio formula for continuous actions, which depends on the squared Mahalanobis distances, to accurately compute advantage estimation in GRPO, ensuring stable gradient propagation during reinforcement learning.

Abstract

Recent MLLMs increasingly rely on visual chain-of-thought to perform region-grounded reasoning over images. However, existing approaches ground regions via either textified coordinates (causing modality mismatch and semantic fragmentation) or fixed-granularity patches that both limit precise region selection and often require non-trivial architectural changes. In this paper, we propose Numerical Visual Chain-of-Thought (NV-CoT), a framework that enables MLLMs to reason over images using continuous numerical coordinates. NV-CoT expands the MLLM action space from discrete vocabulary tokens to a continuous Euclidean space, allowing models to directly generate bounding-box coordinates as actions with only minimal architectural modification. The framework supports both supervised fine-tuning and reinforcement learning. In particular, we replace categorical token policies with a Gaussian (or Laplace) policy over coordinates and introduce stochasticity via reparameterized sampling, making NV-CoT fully compatible with GRPO-style policy optimization. Extensive experiments on three benchmarks against 11 representative visual reasoning baselines demonstrate that NV-CoT significantly improves localization precision and final answer accuracy, while also accelerating training convergence, validating the effectiveness of continuous-action visual reasoning in MLLMs. Project page: https://kesenzhao.github.io/NV-CoT/

Sources

Related papers