Large Discrete Policy: Advancing Explicit Behavior Modeling with Stochastic Iterative Scoring
summary
The gist
Large Discrete Policy (LDiP) is a fully discrete behavior modeling framework that selects actions from a large vocabulary of physically plausible candidates by using stochastic iterative scoring…
In short
Large Discrete Policy (LDiP) is a discrete behavior modeling framework that selects actions from a large set of plausible candidates using stochastic iterative scoring. It improves expressivity by progressively re-scoring and pruning candidates, offering an interpretable decision process that outperforms continuous and strong discrete models across driving, robotics, and vision tasks.
Key concepts
- Stochastic Iterative Scoring
- This mechanism perturbs the scores of action candidates with noise sampled from a Gumbel distribution during training. This encourages the model to explore different plausible actions among candidates without corrupting the discrete vocabulary or generating physically impossible actions, thus improving exploration.
- Large Discrete Vocabulary Construction
- The process starts by clustering offline expert demonstrations into fixed-horizon action chunks using K-Means. This creates a large, discrete vocabulary of action candidates. The iterative scoring then prunes this large set down to a single executable action through progressive selection based on these scores.
- Explicit Decision Process
- LDiP makes the decision process transparent by showing exactly which action candidates are considered at each stage, how they are ranked, and the final selection method. This contrasts with black-box models, making the behavior modeling more interpretable and easier to analyze.
Terminology used across episodes
This episode discusses
- Large Discrete Policy: Advancing Explicit Behavior Modeling with Stochastic Iterative Scoring · Paper Radio
- Qwen3-VL Technical Report
- Qwen2.5-VL Technical Report
- GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
- pi 0: A Vision-Language-Action Flow Model for General Robot Control
- RT-1: Robotics Transformer for Real-World Control at Scale
- VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning
- DriveFine: Refining-Augmented Masked Diffusion VLA for Precise and Robust Driving
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- pi 0.5: a Vision-Language-Action Model with Open-World Generalization
- Categorical Reparameterization with Gumbel-Softmax
- OpenVLA: An Open-Source Vision-Language-Action Model
- Adam: A Method for Stochastic Optimization
- Hydra-MDP++: Advancing End-to-End Driving via Expert-Guided Hydra-Distillation
- LaViDa: A Large Diffusion Language Model for Multimodal Understanding
- DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving
- ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving
- Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillation
- Hydra-NeXt: Robust Closed-Loop Driving with Open-Loop Training
- Zero-Human Demonstration End-to-end Autonomous Driving with Trajectory Scorer
- Generalized Trajectory Scoring for End-to-end Multimodal Planning
The paper
Large Discrete Policy: Advancing Explicit Behavior Modeling with Stochastic Iterative Scoring · Read on arXiv
Fudan University · NVIDIA
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: We're looking at the paper: "Large Discrete Policy".
Dev: Here's the gist of the paper. Large Discrete Policy (LDiP) is a fully discrete behavior modeling framework that selects actions from a large vocabulary of physically plausible candidates by using stochastic iterative scoring, which offers an expressive, plausible, and interpretable alternative to continuous generative models <ref:2609.07049#pg2>.
Rosa: This approach consistently outperforms strong discrete and continuous baselines across various tasks in autonomous driving, robotic manipulation, and vision-language-action settings <ref:2609.07049#pg2>.
Dev: The Core Concept of LDiP LDiP is presented as a fully discrete behavior modeling framework that improves expressivity through stochastic iterative scoring <ref:2609.07049#pg4>.
Rosa: Rather than perturbing actions, LDiP progressively re-scores and prunes candidates with score-space stochasticity, enabling fine-grained ranking and exploration among plausible actions while preserving an explicit decision process <ref:2609.07049#pg4>.
Paper summary: Rosa: So we're looking at this paper called Large Discrete Policy: Advancing Explicit Behavior Modeling with Stochastic Iterative Scoring. The main idea here is they're moving away from these continuous generative models that are hard to understand and often produce actions that just don't make sense physically.
Dev: Right, it suggests a fully discrete framework where the AI picks actions from a big list of candidates that are already plausible, instead of just trying to smooth out continuous movement.
Taro: What I find interesting is how they achieve this expressivity without getting lost in those complex iterative denoising processes you see in other models.
Rosa: They do this by using stochastic iterative scoring, which means instead of just tweaking the actions themselves, they progressively re-score and prune candidates based on some randomness.
Dev: So it’s not about injecting noise into the actions directly, but rather perturbing the scores during training to encourage exploration among those plausible action candidates.
Taro: That score-space perturbation is key because it allows them to explore different options while keeping the final decision process explicit, showing exactly which candidates are being considered and how they're ranked.
Rosa: Exactly. They start by building a large action vocabulary by clustering offline demonstrations, and then they iteratively prune that vocabulary down until only one executable action remains.
Dev: And they introduce this mechanism where before selecting the top k at intermediate stages, the original scores get perturbed with noise sampled from a Gumbel distribution to keep things exploring.
Taro: It keeps the vocabulary discrete and physically plausible because they aren't corrupting the action space, but instead encouraging movement through score-space uncertainty.
Rosa: The paper claims this approach consistently outperforms both strong discrete and continuous baselines across several different tasks, including autonomous driving, robotic manipulation, and vision language action settings.
Dev: That’s a broad claim. I'm interested in where it actually holds up when the system is running in real-time loops with high latency or unexpected world misbehavior.
Taro: I think the iterative pruning process shows diminishing returns after three stages, which tells us that while it's effective early on, you can't just keep adding more pruning steps and expect huge gains.
Rosa: That’s a good point about efficiency. They also show that LDiP avoids the expensive iterative denoising over many steps compared to something like Diffusion Policy, leading to much faster inference speed while improving performance metrics.
Dev: So we're looking at a trade-off here: you get better accuracy and expressivity by using this discrete scoring process, but you also get a more efficient runtime because it doesn't require those long denoising sequences.
Taro: The qualitative results show that early stages cover diverse plausible futures, but later stages concentrate on a smaller set of scene-consistent trajectories as the process refines.
Rosa: In autonomous driving specifically, they demonstrate end-to-end planning robustness by maintaining smooth lane following and decent awareness in complex urban settings.
Dev: But I’m still curious about the limitations they mention regarding manipulation tasks; they admit LDiP falls behind Diffusion Policy in those high degree of freedom scenarios.
Taro: They also flag that under extreme closed-loop conditions, there’s still room for improvement, which points toward the need for more rigorous safety monitoring and human oversight in practical deployment.
Rosa: Overall, what this paper does is provide a fully discrete behavior modeling framework that offers a way to get expressivity without sacrificing interpretability or physical plausibility.
Dev: So, to summarize the core claim of Large Discrete Policy: Advancing Explicit Behavior Modeling with Stochastic Iterative Scoring, it’s about using explicit iterative scoring over a discrete vocabulary to bridge the gap between expressive generative models and interpretable decision-making.
Taro: It makes the process transparent by showing exactly how candidates are ranked and selected, which is important when you need to verify why an AI chose a specific action in a critical situation.
Rosa: And they extend this into LDiP-VLA by integrating it with vision language models, showing it works across different modalities for behavior modeling.
Dev: That extension suggests this framework might be a transferable interface rather than just an architecture-specific trick, which is encouraging for broader AI applications.
Taro: It’s definitely a step toward making these complex planning systems more reliable because you can actually see the decision path being taken before it commits to something.
Rosa: So, thinking about the title and authors of this paper, Large Discrete Policy: Advancing Explicit Behavior Modeling with Stochastic Iterative Scoring by Li, Chang, Sun, Chen, Yao, Shen, Jiang, Wu and Lan means they’re focusing on making behavior modeling both more explicit and more robust through this iterative scoring method.
Dev: It feels like they’re tackling the interpretability problem head-on by forcing the model to operate within a discrete set of candidates guided by a stochastic process.
Taro: For someone who just listens to the show, it means that when you see an AI make a complex driving decision, this paper suggests there’s an underlying, traceable selection path through these plausible actions.
Rosa: That's the practical implication: instead of a black box generating continuous motion, we get a system that picks from a list of known good moves step-by-step and shows us the reasoning.
Conclusion: Rosa: So, to wrap up, this paper is about taking those complex behavior models and making them fully discrete by using this iterative scoring thing they call stochastic iterative scoring.
Dev: Right, so it’s moving away from those messy continuous models and forcing the AI to pick from a specific list of plausible actions at each step.
Rosa: That explicit process is what they emphasize; you can see exactly which candidates are being considered and how they're ranked throughout the decision-making loop.
Dev: It’s a way to keep things interpretable, even when the underlying model is trying to be expressive like a continuous one would be.
Rosa: The authors, Li and his team, are showing that this method works well across driving robots and even when you throw vision language models into it.
Dev: That extension into vision language action settings suggests this isn't just some narrow trick for driving; it’s a more general way to model behavior.
Rosa: They show that the results consistently beat both discrete and continuous methods in autonomous driving tasks, which is pretty solid performance.
Dev: But we have to remember they also tested manipulation tasks, and there are still some areas where this method doesn't quite match the top diffusion policy results there.
Rosa: True, they admit it falls behind diffusion policy on certain high-degree-of-freedom manipulation challenges, so you can’t expect perfection everywhere yet.
Dev: What this means for us in the real world is that we get a much faster way to plan because you aren't running those super long iterative denoising sequences.
Rosa: Plus, they manage to keep the model size manageable while still using a bigger action vocabulary and only three steps of selection.
Dev: So it’s about finding that sweet spot between having enough planning accuracy and keeping the runtime fast enough for real-time control loops.
Rosa: The authors conclude this is a practical way to get behavior modeling that you can actually trust because you have a traceable decision path.
Dev: It's definitely giving us a more transparent system, but we still need those rigorous closed-loop tests and human oversight before we put it on the road.
Rosa: Exactly. So, next up, let's look at how this specific iterative pruning works to narrow down those action candidates.
More episodes
- 2608.13438-ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models
- 2610.34593-Future Prediction as Training-Only Video Supervision for Pretrained VLA Policies
- 2610.35385-EmbodiedLab: Beyond Manipulation, Benchmarking Embodied Agents That Act to Know
- 2610.36378-Data-Driven Hamiltonian for Neural Optimal Control
- 2610.37275-A Heuristic for Scheduling Autonomous Vehicles at Intersections
- 2610.37288-Dynamic Coordination of Autonomous Vehicles at Intersections
- 2610.37632-Real-Time Coordination of Autonomous Vehicles at Intersections
- 2611.13779-Softness is something you also need! Why compliance helps control: information metrics of a passive walker with compliant ankle
- 2611.13411-Residual-Corrected Physics-Informed Neural Network KKL Observers with Input-to-State Stability Guarantees
- 2611.14488-Is Your Robot's Memory Just a Clock? Temporal Shortcuts in Memory-Augmented VLA Policies