Large Discrete Policy: Advancing Explicit Behavior Modeling with Stochastic Iterative Scoring

arXiv:2609.07049 · cs.RO, cs.CV · Submitted 2026-09-07 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "Large Discrete Policy".

Dev: Here's the gist of the paper. Large Discrete Policy (LDiP) is a fully discrete behavior modeling framework that selects actions from a large vocabulary of physically plausible candidates by using stochastic iterative scoring, which offers an expressive, plausible, and interpretable alternative to continuous generative models <ref:2609.07049#pg2>.

Rosa: This approach consistently outperforms strong discrete and continuous baselines across various tasks in autonomous driving, robotic manipulation, and vision-language-action settings <ref:2609.07049#pg2>.

Dev: The Core Concept of LDiP LDiP is presented as a fully discrete behavior modeling framework that improves expressivity through stochastic iterative scoring <ref:2609.07049#pg4>.

Rosa: Rather than perturbing actions, LDiP progressively re-scores and prunes candidates with score-space stochasticity, enabling fine-grained ranking and exploration among plausible actions while preserving an explicit decision process <ref:2609.07049#pg4>.

Paper summary: Rosa: So we're looking at this paper called Large Discrete Policy: Advancing Explicit Behavior Modeling with Stochastic Iterative Scoring. The main idea here is they're moving away from these continuous generative models that are hard to understand and often produce actions that just don't make sense physically.

Dev: Right, it suggests a fully discrete framework where the AI picks actions from a big list of candidates that are already plausible, instead of just trying to smooth out continuous movement.

Taro: What I find interesting is how they achieve this expressivity without getting lost in those complex iterative denoising processes you see in other models.

Rosa: They do this by using stochastic iterative scoring, which means instead of just tweaking the actions themselves, they progressively re-score and prune candidates based on some randomness.

Dev: So it’s not about injecting noise into the actions directly, but rather perturbing the scores during training to encourage exploration among those plausible action candidates.

Taro: That score-space perturbation is key because it allows them to explore different options while keeping the final decision process explicit, showing exactly which candidates are being considered and how they're ranked.

Rosa: Exactly. They start by building a large action vocabulary by clustering offline demonstrations, and then they iteratively prune that vocabulary down until only one executable action remains.

Dev: And they introduce this mechanism where before selecting the top k at intermediate stages, the original scores get perturbed with noise sampled from a Gumbel distribution to keep things exploring.

Taro: It keeps the vocabulary discrete and physically plausible because they aren't corrupting the action space, but instead encouraging movement through score-space uncertainty.

Rosa: The paper claims this approach consistently outperforms both strong discrete and continuous baselines across several different tasks, including autonomous driving, robotic manipulation, and vision language action settings.

Dev: That’s a broad claim. I'm interested in where it actually holds up when the system is running in real-time loops with high latency or unexpected world misbehavior.

Taro: I think the iterative pruning process shows diminishing returns after three stages, which tells us that while it's effective early on, you can't just keep adding more pruning steps and expect huge gains.

Rosa: That’s a good point about efficiency. They also show that LDiP avoids the expensive iterative denoising over many steps compared to something like Diffusion Policy, leading to much faster inference speed while improving performance metrics.

Dev: So we're looking at a trade-off here: you get better accuracy and expressivity by using this discrete scoring process, but you also get a more efficient runtime because it doesn't require those long denoising sequences.

Taro: The qualitative results show that early stages cover diverse plausible futures, but later stages concentrate on a smaller set of scene-consistent trajectories as the process refines.

Rosa: In autonomous driving specifically, they demonstrate end-to-end planning robustness by maintaining smooth lane following and decent awareness in complex urban settings.

Dev: But I’m still curious about the limitations they mention regarding manipulation tasks; they admit LDiP falls behind Diffusion Policy in those high degree of freedom scenarios.

Taro: They also flag that under extreme closed-loop conditions, there’s still room for improvement, which points toward the need for more rigorous safety monitoring and human oversight in practical deployment.

Rosa: Overall, what this paper does is provide a fully discrete behavior modeling framework that offers a way to get expressivity without sacrificing interpretability or physical plausibility.

Dev: So, to summarize the core claim of Large Discrete Policy: Advancing Explicit Behavior Modeling with Stochastic Iterative Scoring, it’s about using explicit iterative scoring over a discrete vocabulary to bridge the gap between expressive generative models and interpretable decision-making.

Taro: It makes the process transparent by showing exactly how candidates are ranked and selected, which is important when you need to verify why an AI chose a specific action in a critical situation.

Rosa: And they extend this into LDiP-VLA by integrating it with vision language models, showing it works across different modalities for behavior modeling.

Dev: That extension suggests this framework might be a transferable interface rather than just an architecture-specific trick, which is encouraging for broader AI applications.

Taro: It’s definitely a step toward making these complex planning systems more reliable because you can actually see the decision path being taken before it commits to something.

Rosa: So, thinking about the title and authors of this paper, Large Discrete Policy: Advancing Explicit Behavior Modeling with Stochastic Iterative Scoring by Li, Chang, Sun, Chen, Yao, Shen, Jiang, Wu and Lan means they’re focusing on making behavior modeling both more explicit and more robust through this iterative scoring method.

Dev: It feels like they’re tackling the interpretability problem head-on by forcing the model to operate within a discrete set of candidates guided by a stochastic process.

Taro: For someone who just listens to the show, it means that when you see an AI make a complex driving decision, this paper suggests there’s an underlying, traceable selection path through these plausible actions.

Rosa: That's the practical implication: instead of a black box generating continuous motion, we get a system that picks from a list of known good moves step-by-step and shows us the reasoning.

Conclusion: Rosa: So, to wrap up, this paper is about taking those complex behavior models and making them fully discrete by using this iterative scoring thing they call stochastic iterative scoring.

Dev: Right, so it’s moving away from those messy continuous models and forcing the AI to pick from a specific list of plausible actions at each step.

Rosa: That explicit process is what they emphasize; you can see exactly which candidates are being considered and how they're ranked throughout the decision-making loop.

Dev: It’s a way to keep things interpretable, even when the underlying model is trying to be expressive like a continuous one would be.

Rosa: The authors, Li and his team, are showing that this method works well across driving robots and even when you throw vision language models into it.

Dev: That extension into vision language action settings suggests this isn't just some narrow trick for driving; it’s a more general way to model behavior.

Rosa: They show that the results consistently beat both discrete and continuous methods in autonomous driving tasks, which is pretty solid performance.

Dev: But we have to remember they also tested manipulation tasks, and there are still some areas where this method doesn't quite match the top diffusion policy results there.

Rosa: True, they admit it falls behind diffusion policy on certain high-degree-of-freedom manipulation challenges, so you can’t expect perfection everywhere yet.

Dev: What this means for us in the real world is that we get a much faster way to plan because you aren't running those super long iterative denoising sequences.

Rosa: Plus, they manage to keep the model size manageable while still using a bigger action vocabulary and only three steps of selection.

Dev: So it’s about finding that sweet spot between having enough planning accuracy and keeping the runtime fast enough for real-time control loops.

Rosa: The authors conclude this is a practical way to get behavior modeling that you can actually trust because you have a traceable decision path.

Dev: It's definitely giving us a more transparent system, but we still need those rigorous closed-loop tests and human oversight before we put it on the road.

Rosa: Exactly. So, next up, let's look at how this specific iterative pruning works to narrow down those action candidates.

Fudan University · NVIDIA

cs.RO, cs.CV

Submitted: 2026-09-07

Updated: 2026-10-31

Project page: https://zhenxinli.net/LargeDiscretePolicy

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 99/100

The gist: Large Discrete Policy (LDiP) is a fully discrete behavior modeling framework that selects actions from a large vocabulary of physically plausible candidates by using stochastic iterative scoring…

Key concepts

Stochastic Iterative Scoring
This mechanism perturbs the scores of action candidates with noise sampled from a Gumbel distribution during training. This encourages the model to explore different plausible actions among candidates without corrupting the discrete vocabulary or generating physically impossible actions, thus improving exploration.
Large Discrete Vocabulary Construction
The process starts by clustering offline expert demonstrations into fixed-horizon action chunks using K-Means. This creates a large, discrete vocabulary of action candidates. The iterative scoring then prunes this large set down to a single executable action through progressive selection based on these scores.
Explicit Decision Process
LDiP makes the decision process transparent by showing exactly which action candidates are considered at each stage, how they are ranked, and the final selection method. This contrasts with black-box models, making the behavior modeling more interpretable and easier to analyze.

Terminology

Summary

Large Discrete Policy (LDiP) is a fully discrete behavior modeling framework that selects actions from a large vocabulary of physically plausible candidates by using stochastic iterative scoring, which offers an expressive, plausible, and interpretable alternative to continuous generative models <ref:2609. p. 007049#pg2>. This approach consistently outperforms strong discrete and continuous baselines across various tasks in autonomous driving, robotic manipulation, and vision-language-action settings <ref:2609. p. 007049#pg2>.

The Core Concept of LDiP

LDiP is presented as a fully discrete behavior modeling framework that improves expressivity through stochastic iterative scoring <ref:2609.07049#pg4>. Rather than perturbing actions, LDiP progressively re-scores and prunes candidates with score-space stochasticity, enabling fine-grained ranking and exploration among plausible actions while preserving an explicit decision process <ref:2609. p. 007049#pg4>. This design makes the decision process explicit by revealing which candidates are considered, how they are ranked, and how the final action is selected <ref:2609. p. 107049#pg4>. The goal is to establish fully discrete action scoring as a general and expressive framework for behavior modeling <ref:2609. p. 107049#pg4>.

Vocabulary Construction and Iterative Pruning

The process begins by constructing a large action vocabulary by clustering offline demonstrations <ref:2609.07049#pg4>. This involves segmenting expert behaviors into fixed-horizon action chunks and applying K-Means clustering to construct a large discrete vocabulary V with K action candidates <ref:2609. p. 307049#pg4>. The iterative scoring then proceeds by progressively pruning candidates according to their scores until a single executable action remains, as shown in Figure 1 (c) <ref:2609. p. 107049#pg4>. This iterative process preserves a fully discrete decision process while allowing the model to progressively prune the vocabulary to the most promising regions of the action space <ref:2609. p. 307049#pg4>.

The Stochastic Scoring Mechanism

To further address uncertainty and approximation errors in the scoring model, LDiP introduces stochastic scoring, a novel mechanism that perturbs the scores during training rather than injecting noise into actions <ref:2609.07049#pg4>. Specifically, before top-k selection at intermediate stages, the original scores are perturbed with noise sampled from a Gumbel distribution <ref:2609. p. 407049#pg4>. This score-space perturbation encourages exploration among plausible action candidates without corrupting the discrete vocabulary or producing physically implausible action candidates <ref:2609. p. 407049#pg4>.

Performance Across Domains

LDiP is benchmarked across various behavior modeling tasks, consistently outperforming both discrete and continuous baselines in autonomous driving, while achieving performance stronger than or on par with the standard Diffusion Policy in high-DoF manipulation tasks <ref:2609.07049#pg4>. When extended with Vision-Language models (VLMs), LDiP-VLA outperforms prior approaches, including diffusion-based and flow-based VLAs <ref:2609. p. 207049#pg4>. The results show that LDiP provides a transferable action modeling interface, rather than an architecture-specific enhancement <ref:2609. p. 607049#pg4>.

Ablation and Efficiency Analysis

Ablation studies confirm the benefits of each component; for instance, adding stochastic scoring achieves the best EPDMS of 87.8 <ref:2609. p. 007049#pg4>. In terms of efficiency, LDiP avoids expensive iterative denoising over many steps compared to Diffusion Policy, resulting in substantially higher inference speed while also improving EPDMS <ref:2609.07049#pg4>. The model maintains a comparable size with the V2-99 backbone while using a larger action vocabulary and three lightweight selection steps <ref:2609. p. 407049#pg4>. This indicates that the proposed multi-step action selection provides a favorable trade-off between planning accuracy and runtime efficiency <ref:2609. p. 607049#pg4>. The iterative pruning process shows diminishing returns after three stages, suggesting that additional pruning stages bring diminishing returns <ref:2609. p. 707049#pg4>.

Qualitative Results and Limitations

Qualitative results visualize the iterative scoring process, showing that at early stages, the remaining vocabulary covers diverse plausible futures while later stages progressively concentrate on a smaller set of scene-consistent trajectories <ref:2609.07049#pg4>. In autonomous driving, LDiP demonstrates end-to-end planning robustness by maintaining smooth lanefollowing behavior and reasonable interaction awareness in complex urban environments <ref:2609. p. 807049#pg4>. However, LDiP falls behind Diffusion Policy on certain manipulation tasks and leaves room for improvement under extreme closed-loop conditions <ref:2609. p. 807049#pg4>. Practical use should therefore require rigorous closed-loop testing, safety monitoring, and human oversight <ref:2609. p. 607049#pg4>. The paper concludes that LDiP provides a practical and interpretable alternative for behavior modeling <ref:2609. p. 807049#pg4>.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. The improved system can generate physically plausible actions by selecting from a large vocabulary of candidates rather than relying on continuous generative models, which are prone to producing physically implausible actions.

  2. The system gains better interpretability by using stochastic iterative scoring, which makes the decision process explicit by revealing which candidates are considered, how they are ranked, and how the final action is selected.

  3. The system can handle environmental uncertainty more robustly because it uses stochastic scoring, which involves perturbing scores during training to encourage exploration among plausible candidates without corrupting the discrete vocabulary or producing physically implausible action candidates.

  4. The system achieves better performance in vision-language-action settings by transferring its scoring mechanism, as LDiP-VLA outperforms prior approaches, including diffusion-based and flow-based VLAs, achieving a PDMS of 92.1 on Navtest v1.

  5. The system can be used for real-time decision making with improved efficiency, as it avoids expensive iterative denoising over 100 DDPM steps compared to Diffusion Policy, running at 12.1 FPS while achieving the best EPDMS score of 87.8 on a single NVIDIA A100 GPU.

  6. The system can reason about occluded agents and environmental uncertainty by progressively concentrat[ing] on a smaller set of scene-consistent trajectories during the iterative scoring process, leading to conservative and context-aware actions in complex urban environments.

  7. The system can be adapted for robotic manipulation tasks, achieving success rates up to 0.99 on Lift and 0.94 on Can, and showing improvements over Diffusion Policy on ToolHang (reaching 0.52 success rate).

Sources

Related papers