SpatialOPSD: Self-Distilling Spatial Intelligence from Verified Coding Agent Traces

arXiv:2610.11366 · cs.LG · Submitted 2026-10-08 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "SpatialOPSD: Self-Distilling Spatial Intelligence from Verified Coding Agent Traces".

Jane: The gist The SpatialOPSD framework internalizes spatial reasoning into Multimodal Large Language Models by distilling verified agent traces into privileged information for on-policy self-distillation,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So we're looking at this paper today, it's called "SpatialOPSD: Self-Distilling Spatial Intelligence from Verified Coding Agent Traces". It sounds super technical, but basically, they’re trying to get spatial reasoning—that tricky part where you figure out three dee shapes from a flat picture—inside large language models without needing those external tools when they're actually making the final answer <ref:2610.11366#pg1>.

Jane: That's right. The core idea here is that instead of relying on an AI using code or some kind of tool to figure out the spatial stuff, they’re feeding the model verified traces from that agent directly into its own training process so it learns it internally.

Lu: It’s about taking something that works well when you give the model a bunch of steps and code execution and turning that external help into something the model can use on its own during its regular training.

Meng: So, if I'm getting this right, instead of running some extra spatial agent every time a user asks a spatial question, we distill that process into the main LLM’s weights so it just *knows* how to reason spatially.

Lalam: Exactly. It’s about turning inference-time benefits into training supervision by taking those verified agent traces and making them privileged information for an on-policy self-distillation method called OPSD.

The paper's summary: Tom: Okay, so what does this SpatialOPSD framework actually do? It starts by saying that conditioning a model on summarized execution traces from a spatial coding agent naturally unlocks the model’s internal chain of thought when it comes to spatial tasks.

Jane: That makes sense. Usually, generic chain-of-thought prompting just gives the model some steps, but this method uses traces from an actual code execution process to guide its thinking on things like three dee perception <ref:2610.11366#pg1>.

Lu: The paper introduces SpatialOPSD as the framework that takes those verified agent traces and turns them into privileged information for OPSD, which is an on-policy self-distillation approach. It’s designed to internalize that tool-based spatial reasoning into a standalone MLLM, meaning it can operate entirely tool-free at inference time.

Meng: That's a big deal for practical deployment. If we can distill this capability, the model doesn't need to run an external code interpreter every time, which cuts down on latency and infrastructure needs significantly.

Lalam: And they handle that tricky part where you don't want the model just to memorize the tool artifacts. They pair this privileged setup with Repetition-Aware Distillation, which penalizes verbatim copying of content or self-repetition during distillation.

The paper's improvements: Tom: The authors are pretty clear about why they did this. They’re addressing the problem that standard chain-of-thought prompting has mixed results for spatial reasoning in MLLMs, so they ask if execution-grounded guidance can actually make a better difference.

Jane: They compare four different inference settings on shared benchmarks, and they show that using a Spatial Coding Agent to iteratively generate and execute code produces an Agent-Trace-Conditioned CoT that is better than just prompting with the agent's direct answers.

Lu: They also look at how summarizing those traces works—they compare FULL-TRACE, SUMMARY, FACT, and INTENT forms of privileged information. And they find that using SUMMARY often gives the best performance with a moderate gap between the teacher and student models during distillation.

Meng: From an engineering standpoint, they found that combining masked distillation with unlikelihood regularization substantially reduces this problem of privileged information leakage compared to just using vanilla OPSD, which is important because we can’t afford for the model to just memorize those trace artifacts.

Lalam: And they show that Step Numbering in the reasoning style actually gave them a big gain, with plus fifty-three point zero points in coverage gain, suggesting that learning a more structured way of thinking is beneficial here.

Conclusion: Tom: So to wrap up on this SpatialOPSD paper, the main point is that they’ve successfully shown how to take tool-based spatial reasoning and distill it into a standalone MLLM without needing the tools at inference time. It’s a self-distillation loop for geometric reasoning.

Jane: It moves beyond just using external tools by taking that agentic benefit and baking it into the model's training data itself, which is a very practical way to enhance capability alongside scaling up model size.

Lu: The implication here is that we can ground inference strictly in observable visual cues, which helps mitigate relying on those privileged training artifacts that could lead to hallucinating unavailable tool traces.

Meng: For me, the most tangible thing is how this removes the reliance on a separate tool execution loop during actual deployment, making the system much more efficient and stable.

Lalam: It really establishes this self-distillation loop for geometric reasoning, showing that we can achieve better spatial understanding by learning from the traces generated by an agent.

Tom: That’s it for SpatialOPSD today. It shows a clear path toward building models with stronger standalone spatial reasoning, and we'll be watching how this affects the next generation of multimodal AI.

Rongxue Li, Meng Yang, Yiru Mao, Yongliang Tao, Lulu Hu, Bin Yang, Zhao Xu, Weihua Luo

Alibaba Group

cs.LG

Submitted: 2026-10-08

Updated: 2026-10-08

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 88/100

The gist: The gist The SpatialOPSD framework internalizes spatial reasoning into Multimodal Large Language Models by distilling verified agent traces into privileged information for on-policy

Key concepts

Spatial Reasoning and the Problem: Spatial reasoning is hard for AI because it requires understanding 3D properties from 2D images. Current methods struggle because verifying intermediate geometric steps is difficult, making standard training brittle and explicit Chain-of-Thought supervision too expensive to scale.
Spatial Coding Agents and Agent Traces: This approach frames spatial reasoning as an agent task where a coding agent writes and executes code iteratively to solve a problem. The resulting trace ($ au$) captures the entire reasoning process, providing verifiable, step-by-step data that can be used for training.
Repetition-Aware Distillation: This technique prevents the model from simply copying privileged information during self-distillation. It uses repetition masking and unlikelihood regularization to penalize tokens that repeat previous content or verbatim privileged traces, ensuring the model learns general reasoning instead of memorizing data.

Terminology

Trace — 1× in this paper · explained in 2 episodes

Summary

The gist The SpatialOPSD framework internalizes spatial reasoning into Multimodal Large Language Models by distilling verified agent traces into privileged information for on-policy self-distillation, achieving superior performance over standard training paradigms.

Spatial Reasoning and the Problem

Spatial reasoning requires inferring latent 3D properties from 2D observations, posing a fundamental challenge for multimodal large language models (MLLMs) While final answers may be verifiable, the intermediate geometric inferences are difficult to annotate or validate automatically. Answer-level supervised fine-tuning (SFT) forces spatial understanding to remain implicit and brittle, while explicit Chain-of-Thought (CoT) supervision is prohibitively expensive to scale. The iterative loop of decoding and tool execution introduces high latency and infrastructure overhead.

Spatial Coding Agents and Agent Traces

Recent methods cast spatial reasoning as an agentic task, where a spatial coding agent operates over a persistent Python kernel equipped with perception modules and scientific libraries. Given an input x, the agent iteratively writes and executes code to produce a T-step trace: τ = (r0, a0, o0... rt, at, ot..., yA), where rt is a natural-language reasoning step. This paradigm elegantly resolves the aforementioned verifiability bottleneck.

SpatialOPSD Framework

SpatialOPSD is an on-policy self-distillation framework that internalizes spatial reasoning into a standalone MLLM by formulating verified agent traces as privileged information. The starting observation is that conditioning an MLLM on spatial coding agent traces elicits higher-quality CoT reasoning on spatial tasks. SpatialOPSD converts this inferencetime benefit into training supervision by summarizing verified agent traces into privileged information for OPSD.

Repetition-Aware Distillation

To mitigate privileged-information leakage during distillation, Repetition-Aware Distillation combines repetition masking with unlikelihood regularization. This involves detecting spans that either repeat earlier rollout content or transcribe privileged information and applying an unlikelihood penalty to exclude them from the standard distillation supervision. The final objective is Lrep-distill = Ey∼pθ¯(·x) [ldistill(y) + λrep lrep(y)], where ldistill uses a mask mt to exclude flagged tokens and lrep applies an unlikelihood penalty when mt=1.

Empirical Results and Contributions

Experiments across multiple benchmarks demonstrate that self-distilling SpatialOPSD achieves higher average accuracy than SFT and GRPO on both spatial and OOD datasets, exhibiting superior performance and generalization. SpatialOPSD approaches the performance of tool-using spatial agents while operating entirely tool-free. The contributions include showing that spatial coding-agent traces effectively elicit multimodal spatial reasoning, proposing SpatialOPSD to distill tool-augmented spatial reasoning into multimodal models, and introducing Repetition-Aware Distillation to reduce privileged-information leakage during OPSD.

Ablations and Analysis

Ablations show that combining masked distillation (Mask) and unlikelihood regularization (UL) substantially reduces PI leakage relative to vanilla OPSD, while varying Lmin and λrep demonstrates complementary roles in balancing knowledge transfer and leakage suppression. The framework learns to adopt a more structured reasoning style, with Step Numbering exhibiting the largest coverage gain (+53.0 points). Qualitative analysis confirms that SpatialOPSD-9B correctly disentangles multi-axis trajectories, whereas vanilla OPSD suffers from shortcut learning and hallucinating unavailable tool traces.

Conclusion

SpatialOPSD outperforms SFT and RL baselines and approaches tool-using agents without inference-time tools. By using training data generated by the same MLLM, SpatialOPSD forms a self-distillation learning loop, offering a practical route from tool-assisted data generation to stronger standalone spatial reasoning. The framework establishes a self-distillation loop for geometric reasoning.

How it works

Trace Collection involves deploying a spatial coding agent A as an oracle that solves each training query x through a multi-turn code-execution loop, retaining only the verified interaction record τ = (r0, a0, o0..., rT −1, aT −1, oT −1, yA) when the final prediction yA passes deterministic verification.
Privileged Information Construction involves formulating privileged information I = Φ(τ, y) to condition the teacher model, with variants such as FULL-TRACE (retaining τ verbatim), SUMMARY (condensing traces into key reasoning points), FACT (extracting factual findings), and INTENT (extracting the underlying strategy).
Distillation involves using OPSD to distill the teacher’s reasoning into a student policy pθ(· x) via on-policy self-distillation, where the loss is modified by Repetition-Aware Distillation to penalize verbatim copying of privileged content and self-repetition.
The training objective is Lrep-distill = Ey∼pθ¯(·x) [ldistill(y) + λrep lrep(y)], which balances preserving supervision outside detected repetitions with an unlikelihood penalty on flagged tokens.

Ablations and Analysis

The paper evaluates the impact of different privileged information types, showing that SUMMARY achieves the best performance with a moderate teacher–student entropy gap. The use of verification significantly improves performance by 4.9 points on MindCube-Tiny and 0.5 points on ViewSpatial. The framework generalizes across different coding agents, demonstrating agent-agnostic scalability.

Conclusion

SpatialOPSD introduces a self-distillation loop for geometric reasoning that moves beyond tool augmentation by internalizing trace-conditioned benefits into the MLLM's training process. This approach provides a viable axis for capability enhancement alongside parameter scaling. The method successfully grounds inference strictly in observable visual cues, mitigating spurious reliance on privileged training artifacts.

How it works

Trace Collection involves deploying a spatial coding agent A as an oracle that solves each training query x through a multi-turn code-execution loop, retaining only the verified interaction record τ = (r0, a0, o0...

Improvements for AI systems

1 Spatial reasoning internalizing tool-based capabilities: The improved system can operate entirely tool-free at inference time by distilling verified agent traces into privileged information for on-policy self-distillation, achieving performance competitive with tool-using agents while eliminating inference latency and external dependencies.

2 Mitigating privileged information leakage: The Repetition-Aware Distillation framework actively combats verbatim copying of privileged content and self-repetition by applying an unlikelihood penalty to flagged tokens, ensuring the model learns task-relevant reasoning rather than memorizing trace artifacts.

3 Enhanced reasoning grounding: The system generates reasoning grounded in concrete spatial analysis by conditioning Chain-of-Thought on agent traces, resulting in a significant increase in references to 3D perception and counterfactual motion concepts compared to generic CoT prompting.

Abstract

Spatial coding agents significantly improve spatial reasoning in Multimodal Large Language Models (MLLMs) by using external tools to generate verified execution traces. However, this paradigm inherently suffers from prohibitive inference-time overhead and external dependencies. In this paper, we explore whether an MLLM can internalize this agentic capability to operate entirely tool-free. We begin with a simple observation: prompting an MLLM with summarized execution traces of a spatial coding agent naturally unlocks the model's internal spatial Chain-of-Thought (CoT). Motivated by this, we introduce SpatialOPSD, an on-policy self-distillation framework that internalizes spatial reasoning into a standalone MLLM by formulating verified agent traces as privileged information. To mitigate privileged-information leakage during distillation, we introduce Repetition-Aware Distillation, which combines repetition masking with unlikelihood regularization. Experiments across multiple benchmarks demonstrate that self-distilling SpatialOPSD achieves higher average accuracy than SFT and GRPO on both spatial and OOD datasets, exhibiting superior performance and generalization.