NS-VLA: Towards Neuro-Symbolic Vision-Language-Action Models

arXiv:2603.09542 · cs.RO · Submitted 2026-03-10 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "NS-VLA: Towards Neuro-Symbolic Vision-Language-Action Models".

Dev: Here's the gist of the paper. Vision-Language-Action (VLA) models still face structure-blind backbones, backbone-bound generalization, and flat single-objective optimization.

Rosa: To address these challenges, we propose a novel Neuro-Symbolic Vision-Language-Action (NS-VLA) framework.

Dev: This framework introduces a Neuro-Symbolic Encoder for plan-constrained primitive inference, a Neuro-Symbolic Solver that conditions a backbone-agnostic policy on the active primitive, and Hierarchical Joint Policy Optimization with rewardgranularity matching.

Rosa: Experiments on robotic manipulation benchmarks demonstrate that NS-VLA outperforms previous methods in both one-shot training and data-perturbed settings, while simultaneously exhibiting superior zero-shot generalizability and expanded exploration space.

Title and authors: Rosa: So, we’re looking at this paper called "NS-VLA: Towards Neuro-Symbolic Vision-Language-Action Models," and the authors, Ziyue Zhu, Shangyang Wu, Shuai Zhao, and others are tackling some big problems with current vision language action models.

Dev: It sounds like they're aiming to fix things like structure blindness in these VLA models and the problem where generalization gets stuck on whatever backbone you pretrain them with.

Rosa: Right. Basically, they propose a new framework called NS-VLA that introduces a Neuro-Symbolic Encoder for figuring out the plan using primitives, a Neuro-Symbolic Solver to condition the action policy on that active primitive, and this Hierarchical Joint Policy Optimization with reward granularity matching.

Dev: That sounds like they're trying to bridge the gap between high-level symbolic planning and low-level continuous control in a more structured way than what we see now.

Taro: I'm wondering how this structural awareness helps when things go wrong in the real world, you know, when the environment misbehaves.

Rosa: Well, the paper says this Neuro-Symbolic Encoder parses instructions into a plan of primitives and tracks execution using a monotone pointer which exposes that compositional structure as an inductive bias for the backbone-agnostic part of the system.

Dev: So it’s not just letting the model learn everything from scratch; it's giving it these reusable atomic operations, like "pick up object" or "move to location," that are shared across different tasks.

Taro: That sounds promising for robustness because if the world changes unexpectedly, the system can rely on those known atomic steps instead of getting totally lost in a massive end-to-end network.

Rosa: Exactly. And then they have this Neuro-Symbolic Solver that takes the primitive selected by the encoder and conditions a backbone agnostic policy on that active primitive to generate the actual continuous action sequence.

Dev: That means you’ve got a symbolic decision—the primitive—feeding directly into something that doesn't care about the specific pretraining of, say, a large vision model.

Taro: I see how that decouples things; if the perception part is struggling with an instruction it hasn't seen before, the planning layer still has this grounded plan to work with.

Title and authors: Rosa: And then they use Hierarchical Joint Policy Optimization where both parts are updated with reward streams that match their emission profile, so the high-level factor gets sparse milestone rewards and the low-level factor gets dense per step rewards.

Dev: That sounds like a smart way to balance learning; you get high-level guidance on completing the task and low-level shaping for smooth continuous control at every small step.

Taro: I liked that part about matching reward granularity; it seems like the density of feedback is tailored exactly to what each part needs to learn effectively.

Rosa: They show that this NS-VLA framework outperforms previous methods in both one-shot training and when the data gets a little perturbed, but also shows better zero-shot generalizability.

Dev: I saw some numbers on that; they got seventy-six point five percent success rate on Libero in one shot, which is pretty solid for that setup.

Taro: And they showed gains even when the tasks were completely unseen, like on the CALVIN ABC to D chain execution where it reached a ninety-one point two percent success rate across five tasks <ref:2603.09542#pg2>.

Rosa: They also tested this on a real robot arm, the myCobot 320pi, and it got success rates of seventy-six point seven for Pick, and then dropped to about fifty percent for Move and Place.

Dev: That's good data because those are real-world manipulation benchmarks; having that kind of performance across different physical skills is what matters in a practical sense.

Taro: I noticed their error analysis pointed out that the most common failure type was visual grounding failure, where the system knew which primitive to use but couldn't actually see the right object when things got a little noisy.

Rosa: That makes sense; it shows that even with good planning, the perception part can still struggle with spatial attention under environmental perturbations.

Dev: And they also noted execution instability, which happens when the model generates continuous action sequences that just don't work physically, often because of mapping sparse features to high precision control.

Title and authors: Taro: So it’s not just about the planning logic or the perception failing; there’s a final hurdle where the continuous action generation itself becomes unstable during execution.

Rosa: And looking at their limitations, they admitted that this framework is currently bound to a specific vocabulary of primitives, U, meaning if you give it an instruction outside that set, it just falls back to the original instruction.

Dev: Also, they mentioned that the reward weights used in their hierarchical policy are fixed across different training regimes and they only trained for fifty joint iterations with eight rollouts under a Qwen three-B encoder backbone <ref:2603.09542#pg3>.

Taro: So while the structure is strong, it still has those limitations where it doesn't adapt to completely novel instruction formats or learn the optimal learning rates for its own reward shaping.

Rosa: They suggested future work could involve replacing that rule-based extractor with something unsupervised for primitive discovery, or making those reward weights learnable parameters that co-optimize inside the joint loss.

Dev: That would be interesting because if you can make the shaping signal adapt to each task category’s trajectory length, it could really refine how fast the system learns.

Taro: I also saw a mention of augmenting their joint update with a process supervision term, which sounds like it would trade success rate against deployment cost, which is something I care about when we move this out of the lab.

Rosa: So to wrap up on "NS-VLA: Towards Neuro-Symbolic Vision-Language-Action Models," the core idea is using that Neuro-Symbolic Encoder and Solver to ground instructions into primitives, which then feed a backbone agnostic policy, all refined by hierarchical joint optimization with matched reward granularity.

Dev: It shows that explicitly modeling the plan structure can help VLA models generalize better and explore their action space more effectively than monolithic end-to-end approaches.

Taro: It’s definitely a step forward in how we build systems that need to interact with the physical world reliably, even when things get messy.

Rosa: Yeah, it’s a solid piece of research showing how combining symbolic structure with reinforcement learning can yield much more robust performance on manipulation tasks.

The paper's summary: Rosa: So we're looking at this paper called NS-VLA, and it’s all about taking those vision language action models that are getting really big and making them more structured and reliable.

Dev: Basically, they say these current models have problems because they’re structure blind. They don't understand the underlying plan of the task well enough.

Rosa: Exactly. They call this a backbone-bound generalization problem, meaning if you train it on one type of robot or one set of data, it struggles when you try to use that same model for something slightly different.

Dev: And they’re dealing with flat single objective optimization, which means the learning process isn't focused enough on what actually matters in the task.

Rosa: To fix this, they introduce a Neuro-Symbolic Vision Language Action framework. It uses a Neuro-Symbolic Encoder to figure out the plan by parsing instructions into a sequence of primitives.

Dev: So, it’s like giving the model an explicit blueprint before it even tries to move anything. This gives it that structure you mentioned earlier as an inductive bias, making it less reliant on whatever backbones you use.

Rosa: Right. Then they have a Neuro-Symbolic Solver that bridges this symbolic plan with the continuous action backbone. It conditions that policy directly on the active primitive, not just some generic input.

Dev: That’s smart because it decouples the high level planning from the low level control, letting each part do what it’s best at.

Rosa: And they tie it all together with Hierarchical Joint Policy Optimization. They match reward granularity to the policy type. The high-level part gets sparse rewards for big milestones, and the low-level part gets dense rewards for every small step it takes.

Dev: That reward matching is key because it makes sure both parts are learning in a way that fits their job—the planner learns when to switch goals, and the controller learns how to move smoothly between those goals.

Rosa: The results they show are pretty compelling. They say NS-VLA beats previous methods in one-shot training and even when you mess with the data a little bit.

Dev: And for zero shot generalization, it’s way better than VLA Adapter or Simple VLA RL, especially on long horizon chains like that CALVIN ABC to D task.

Rosa: It actually reached ninety-one point two percent success rate across five tasks in one of those tests, which is a big jump over the others they compared it against.

Dev: The real world test on the myCobot arm also shows success rates of seventy-six point seven percent on picking and fifty percent on moving and placing. That’s where you see if this stuff actually moves beyond simulation.

Rosa: But, there are some caveats too. They found that the most common failure was visual grounding failure—the model knew what primitive to use but couldn't focus on the right object when things got visually noisy in the environment.

Dev: And execution instability, where it generates continuous actions that just fail physically, often because it’s trying to map sparse features to high precision control.

Rosa: The limitation they point out is that their primitive vocabulary is fixed. If you give it an instruction outside that set, it just falls back to the original instruction and doesn't parse anything new.

Dev: They also mentioned that the reward weights are fixed across different training regimes, so if you change your training setup a lot, those shaping signals don't automatically adapt to the new task length.

Rosa: They suggest future work involves making those primitive sets discoverable without explicit rules and letting those reward weights become learnable parameters inside the joint loss function.

Dev: That would let the system itself figure out how much emphasis to put on high-level milestones versus low-level progress depending on the specific task it's doing.

Rosa: It’s a lot of work to make that kind of adaptive shaping happen, but if it works, it could make VLA models much more robust for real robot deployment.

The paper's improvements: Rosa: So we’ve seen how NS-VLA uses that Neuro-Symbolic Encoder and Solver to ground instructions into primitives and condition the action backbone on them, all while using reward matching for the high and low levels.

Dev: Right, it's a lot of work to get that structure right in the first place, but those improvements they suggest are really interesting for making this system more flexible.

Rosa: One big suggestion is replacing that fixed rule-based extractor with something unsupervised to discover primitives instead of just relying on a set U of instructions.

Dev: That would mean the system could learn new ways to plan tasks just by looking at instructions, not being locked into what the authors manually put in there initially.

Rosa: And another thing they pointed out is making those reward weights—the lambdas for high level and low level progress—learnable parameters inside the joint loss function.

Dev: So instead of having those weights fixed across different training runs, the model could actually figure out the best shaping signal for each specific task it’s running.

Rosa: That would let the system adapt its learning speed and focus based on what kind of task it is currently doing, which seems like a big step toward true adaptability.

Dev: It moves away from having to tune those hyperparameters manually for every new application, which is something engineers deal with all the time in real-time loops.

Rosa: They also mentioned augmenting the joint update with a process supervision term, which they suggest could trade success rate against deployment cost.

Dev: That means if you want a robot to be super safe and not make mistakes in the field, you could train it to prioritize safety over just getting the highest success score.

Rosa: It’s about learning to balance performance with risk during deployment, which is a key consideration for any physical system.

Dev: So we've seen how they build a strong structural foundation and then these suggestions are about making that foundation smarter and more flexible in terms of its learning signals.

Rosa: It shows they aren't just building a static pipeline; they’re thinking about how the entire learning process can be made more self-optimizing for real-world use.

Conclusion: Rosa: So to wrap up, NS-VLA is essentially about putting a structured planning layer on top of VLA models using symbolic primitives and hierarchical reward matching to get much better generalization outside the lab.

Dev: It's a solid framework for moving from monolithic end-to-end systems toward something that respects the underlying task logic, which is crucial for building any real robot system where you need a safety case.

Taro: I'm just curious if this structural grounding helps when the world throws unexpected stuff at it, like when things get visually noisy or the environment doesn't match what was expected.

Rosa: The paper shows that even with those visual grounding failures, NS-VLA handles them better because it’s grounded in a plan of primitives rather than just raw pixels.

Dev: That’s good for real-time loops because it gives the planner a clear, discrete decision point to fall back on if the continuous control output looks physically wrong.

Taro: It seems like they're making the recovery process more graceful because the high-level factor is guided by those sparse milestones, which keeps it focused on finishing the goal.

Rosa: And for someone listening to this show, it means we might see VLA models that are less fragile when you take them out into a messy factory floor or a real home instead of just a controlled simulation.

Dev: The potential here is reducing the amount of tedious fine-tuning needed when you want an AI to perform a sequence of tasks, because the structure provides the necessary constraints for learning.

Taro: I still think we need more work on that unsupervised primitive discovery part, though; if it can learn its own vocabulary, then it could handle truly novel instructions without us having to pre-define everything.

Rosa: Yeah, that's definitely where the future goes, moving toward systems that are truly general and not just experts in a few specific tasks.

Dev: We’ve got a lot more on these models next week about how they actually handle long horizon planning when the environment is constantly changing, so stick around.

Beijing University of Posts and Telecommunications · Nanyang Technological University

cs.RO

Submitted: 2026-03-10

Updated: 2026-11-11

Comments: Accepted by NeurIPS 2026 main conference

Journal ref: NeurIPS 2026

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 98/100

The gist: Vision-Language-Action (VLA) models still face structure-blind backbones, backbone-bound generalization, and flat single-objective optimization.

Key concepts

Neuro-Symbolic Encoder
This component parses natural language instructions into an ordered sequence of atomic 'primitives' (like 'move' or 'grasp'). It tracks execution using a monotone pointer, effectively exposing the compositional structure of the plan. This provides a backbone-agnostic way for the model to understand and execute complex instructions by breaking them down into reusable steps.
Neuro-Symbolic Solver
This solver bridges the gap between discrete symbolic decisions (the selected primitive) and continuous action generation. It takes the active primitive chosen by the Encoder and combines it with perceptual context to create a 'primitive-conditioned input' for a chunked action backbone. This allows the low-level policy to focus its continuous control actions specifically on executing that required atomic operation.
Hierarchical Joint Policy Optimization
This training method updates two policy factors at different levels using reward streams tailored to their needs. The high-level factor uses sparse, milestone rewards for task completion, while the low-level factor uses dense rewards for per-chunk progress. This 'reward granularity matching' ensures both decision-making and continuous control learn effectively from signals appropriate to their scope.
Backbone-Agnostic Inductive Bias
The framework is designed so that the symbolic structure learned by the Encoder is not tied to a specific underlying VLA backbone. By conditioning the policy on primitives rather than relying solely on a monolithic model, NS-VLA maintains structural awareness regardless of which continuous action network it uses. This allows for better generalization across different underlying models.

Terminology

Summary

Vision-Language-Action (VLA) models still face structure-blind backbones, backbone-bound generalization, and flat single-objective optimization. p. 0

Where the paper says it

NS-VLA: Towards Neuro-Symbolic Vision-Language-Action Models Ziyue Zhu1,∗ Shangyang Wu1,∗ Shuai Zhao1,† Zhiqiu Zhao1 Jian Zhang2 Shengjie Li1 Yi Wang2 Anh Tuan Luu2 Xinliang Zhou2 Fang Li2 Haoran Luo2,† 1Beijing University of Posts and Telecommunications 2Nanyang Technological University Homepage GitHub Model Dataset Abstract Vision-Language-Action (VLA) models are formulated to ground instructions in visual context and generate action sequences for robotic manipulation. Despite recent progress, VLA models still face structure-blind backbones, backbone-bound generalization, and flat single-objective optimization.

To address these challenges, we propose a novel Neuro-Symbolic Vision-Language-Action (NS-VLA) framework. p. 0
Where the paper says it

Despite recent progress, VLA models still face structure-blind backbones, backbone-bound generalization, and flat single-objective optimization. To address these challenges, we propose a novel Neuro-Symbolic Vision-Language-Action (NS-VLA) framework.

This framework introduces a Neuro-Symbolic Encoder for plan-constrained primitive inference, a Neuro-Symbolic Solver that conditions a backbone-agnostic policy on the active primitive, and Hierarchical Joint Policy Optimization with rewardgranularity matching. p. 0
Where the paper says it

To address these challenges, we propose a novel Neuro-Symbolic Vision-Language-Action (NS-VLA) framework. It introduces a Neuro-Symbolic Encoder for plan-constrained primitive inference, a Neuro-Symbolic Solver that conditions a backbone-agnostic policy on the active primitive, and Hierarchical Joint Policy Optimization with rewardgranularity matching.

Experiments on robotic manipulation benchmarks demonstrate that NS-VLA outperforms previous methods in both one-shot training and data-perturbed settings, while simultaneously exhibiting superior zero-shot generalizability and expanded exploration space. p. 0
Where the paper says it

It introduces a Neuro-Symbolic Encoder for plan-constrained primitive inference, a Neuro-Symbolic Solver that conditions a backbone-agnostic policy on the active primitive, and Hierarchical Joint Policy Optimization with rewardgranularity matching. Experiments on robotic manipulation benchmarks demonstrate that NS-VLA outperforms previous methods in both one-shot training and data-perturbed settings, while simultaneously exhibiting superior zero-shot generalizability and expanded exploration space.

Pruning redundant reasoning tokens

The NS-VLA framework is structured around three main components: a Neuro-Symbolic Encoder that grounds the instruction-conditioned history into the active primitive, a Neuro-Symbolic Solver that bridges that primitive to a chunked action backbone, and Hierarchical Joint Policy Optimization that updates both factors with reward streams matched to their granularity. The Neuro-Symbolic Encoder parses instructions into a plan of primitives and tracks execution with a monotone pointer, exposing compositional structure as a backbone-agnostic inductive bias. p. 1

Where the paper says it

These considerations motivate us to propose a Neuro-Symbolic Vision-Language-Action (NS-VLA) framework, as shown in Figure 1, for structure-aware, generalizable, and exploratory manipulation. First, the Neuro-Symbolic Encoder parses instructions into a plan of primitives and tracks execution with a monotone pointer, exposing compositional structure as a backbone-agnostic inductive bias.

This encoder uses a structured-plan extractor P that combines a deterministic grammar parser on the instruction string with a template-driven consistency check against the initial scene to parse x into an ordered sequence of primitives by combining an operation op and its arguments args. p. 3
Where the paper says it

Plan Extraction. Given the instruction x and initial observation o0, a structured-plan extractor P parses x into an ordered sequence of primitives by combining a deterministic grammar parser on the instruction string with a template-driven consistency check against the initial scene, the second route invoked only when the first cannot resolve a slot.

How the training signal is built

The Hierarchical Joint Policy Optimization updates both factors with reward streams whose granularity matches their emission profile. Specifically, the high-level factor selects a primitive per decision and is coupled to a sparse milestone return firing at task completion and primitive transitions, while the low-level factor emits a chunk per decision and is coupled to a dense per-chunk shaping return. p. 4

Where the paper says it

Granularity-Matched Reward Streams. The high-level factor selects a primitive per decision and is coupled to a sparse milestone return firing at task completion and primitive transitions, while the low-level factor emits a chunk per decision and is coupled to a dense per-chunk shaping return: R h (τ ) = r task(τ ) + λsegX t bt, r l t = λprogγ Φt+1 − Φt + λsub r sub t + λsm r sm t + λgr r gr t , (8) where τ is a rollout trajectory, r task ∈ {0, 1} is a terminal success indicator, bt is a segmentboundary flag, Φt is a shaping potential whose telescoping difference drives progress with discount γ ∈ (0, 1], r sub t is a sub-goal reaching indicator, r sm t penalizes consecutive-action change, r gr t aligns gripper state with the active primitive, and λseg, λprog, λsub, λsm, λgr > 0 are mixing weights.

The reward streams are defined as R h(τ) = r task(τ) + λsegX tbt, and r l t = λprogγ Φt+1 − Φt, where the latter term drives progress with discount γ. p. 4
Where the paper says it

Granularity-Matched Reward Streams. The high-level factor selects a primitive per decision and is coupled to a sparse milestone return firing at task completion and primitive transitions, while the low-level factor emits a chunk per decision and is coupled to a dense per-chunk shaping return: R h (τ ) = r task(τ ) + λsegX t bt, r l t = λprogγ Φt+1 − Φt + λsub r sub t + λsm r sm t + λgr r gr t , (8) where τ is a rollout trajectory, r task ∈ {0, 1} is a terminal success indicator, bt is a segmentboundary flag, Φt is a shaping potential whose telescoping difference drives progress with discount γ ∈ (0, 1], r sub t is a sub-goal reaching indicator, r sm t penalizes consecutive-action change, r gr t aligns gripper state with the active primitive, and λseg, λprog, λsub, λsm, λgr > 0 are mixing weights.

The high-level factor is refined under a trajectory-level weight derived from R h, while the low-level factor receives a chunk-level weight derived from r l t. p. 4
Where the paper says it

Bi-Granular Advantage-Weighted Update. Both factors are refined under a advantage-weighted template instantiated at the granularity each support: the high-level factor receives a trajectory-level weight derived from Rh , while the low-level factor receives a chunk-level weight derived from r l t : L h (ϕ) = − Eτi h w h i X t log π h ϕ u (i) t | H(i) t i + βkl DKLπ h ϕ π h,⋆ ϕ , L l (θ) = Et h w l t π l θ (et) − At 1 i , Ljoint(Θ) = L h (ϕ) + λl L l (θ), (9) where w h i ∝ exp(Rh i − µG)/(σG + ε) uses the group mean µG and standard deviation σG of {Rh j } G j=1 over G rollouts with stability constant ε, w l t ∝ exp(Aˆl t/βl) is the chunk weight normalized over the rollout with chunk-level advantage Aˆl t formed from r l t relative to a running baseline, π h,⋆ ϕ is a frozen behavior-cloning anchor with KL coefficient βkl > 0, βl > 0 is the chunk temperature, At is the executed chunk, and λl > 0 mixes the two terms.

Where the gains come from

The superiority of NS-VLA stems from its ability to maintain structural awareness and decouple policy factors. The Neuro-Symbolic Solver bridges the discrete primitive selected by the Encoder to a continuous action backbone by composing the active primitive with the perceptual context into a primitiveconditioned input. p. 3

Where the paper says it

4 and proofs in Appendix B. 2 Neuro-Symbolic Solver The Neuro-Symbolic Solver bridges the discrete primitive selected by the Encoder to a continuous action backbone, by composing the active primitive with the perceptual context into a primitiveconditioned input and then emitting open-loop action chunks executed before the next decision

Furthermore, Hierarchical Joint Policy Optimization updates both factors with reward streams whose granularity matches their emission profile. p. 4
Where the paper says it

5 and proofs in Appendix B. 3 Hierarchical Joint Policy Optimization We refine the trainable parameters Θ = {ϕ, θ} through online interaction: the two factors receive reward streams whose granularity matches their emission profile, and a single advantage-weighted objective couples both factors without trajectory-level density estimation on the continuous chunk.

Ablation studies show that removing the Encoder cuts 1-shot SR from 69. p. 6
Where the paper says it

0 Ablation. As shown in Table 2, removing the Encoder cuts 1-shot SR from 69.

1 to 58.2 and OOD SR from 49. p. 5
Where the paper says it

Bold marks the best column; underline marks the second-best. Method LIBERO (in-distribution, 1-shot) LIBERO-Plus (perturbed, OOD) Robust.

8 to 20.9, while removing the RL stage costs 6. p. 6
Where the paper says it

Removing the Solver costs 5. 6 points; Removing the RL stage costs 6.

4 and 19.8 points.

Generalization and robustness

NS-VLA exhibits superior zero-shot generalizability across different protocols. On the LIBERO-90 dataset, NS-VLA reaches 14. p. 6

Where the paper says it

As shown in Table 1, models trained on the full LIBERO are evaluated under perturbation. NS-VLA reaches 49.

4 average SR, ahead of VLA-Adapter at 10. p. 5
Where the paper says it

NS-VLA reaches 69. 1 average SR, ahead of VLA-Adapter at 65.

2 and SimpleVLA-RL at 8. p. 5
Where the paper says it

0 VLA-Adapter [49] 80. 9 SimpleVLA-RL [29] 65.

5. For long-horizon chain execution on CALVIN ABC→D, NS-VLA reaches 91. p. 6
Where the paper says it

5 Analysis of Generalization (RQ4) As shown in Figure 7, we evaluate generalization under two protocols: zero-shot transfer to ninety unseen tasks on LIBERO-90, and long-horizon chain execution under split-level distribution shift on CALVIN ABC→D. This subsection tests how NS-VLA generalizes to horizons not seen during training.

2 5-task SR and 4. p. 0
Where the paper says it

Training proceeds in two stages: behavior cloning on expert demonstrations [15, 16], then online reinforcement learning that explores beyond the demonstration distribution [35, 29, 18]. The assumption underneath is that given pretraining and capacity, this backbone internalizes the task decomposition, compositional generalization, and hierarchical training signal that long-horizon manipulation requires.

72 average completed chain length, ahead of VLA-Adapter at 80. p. 7
Where the paper says it

2 5-task SR and 4. 72 average completed chain length, ahead of VLA-Adapter at 80.

0 and 4.50 by 11.2 and 0.22 respectively. p. 7
Where the paper says it

72 average completed chain length, ahead of VLA-Adapter at 80. 22 respectively.

In real-world deployment on the myCobot 320pi arm, NSVLA reaches 76. p. 8
Where the paper says it

7 Performance on Real-World Tasks (RQ6) Realsense D435i myCobot 320pi Pick Move Place Figure 10: Real-world workspace setup. As shown in Figure 9, we deploy NS-VLA on a myCobot 320pi arm equipped with two Realsense D435i cameras and evaluate against VLA-Adapter and SimpleVLA-RL across three task categories.

7 on Pick, 60. p. 3
Where the paper says it

Instruction clause Primitive ops place white mug on left plate pick place_on place yellow-white mug on right plate pick place_on place chocolate pudding right of plate pick place_rel put yellow-white mug in microwave pick place_in close microwave close turn on stove turn_on put alphabet soup in basket pick place_in put book in caddy back compartment pick place_in (a) Clause-to-primitive conversion examples. 6% primitive pick place_in place_on close place_rel turn_on open (b) Primitive distribution.

0 on Move, and 50. p. 8
Where the paper says it

0 (a) Per-category SR. Pick Move Place 0 20 40 60 80 Success rate (%) 53 40 30 43 30 20 77 60 50 VLA-Adapter SimpleVLA-RL NS-VLA (Ours) (b) Grouped bars.

0 on Place, ahead of VLA-Adapter by 23. p. 8
Where the paper says it

0 on Move, and 50. 0 on Place, ahead of VLA-Adapter on every category by 23.

4, 20.0, and 20.0 points respectively. p. 7
Where the paper says it

72 average completed chain length, ahead of VLA-Adapter at 80. 22 respectively.

The framework's joint update preserves optimality and KL-bounds the high-level policy drift from its behavior-cloning anchor.

Error Analysis

The error analysis categorizes failures into three main types on the LIBERO-PLUS benchmark. These include Incorrect Primitive Transition, where the primitive classifier fails to predict the correct next sub-goal, leading to a violation of the task’s logical structure. p. 4

Where the paper says it

Granularity-Matched Reward Streams. The high-level factor selects a primitive per decision and is coupled to a sparse milestone return firing at task completion and primitive transitions, while the low-level factor emits a chunk per decision and is coupled to a dense per-chunk shaping return: R h (τ ) = r task(τ ) + λsegX t bt, r l t = λprogγ Φt+1 − Φt + λsub r sub t + λsm r sm t + λgr r gr t , (8) where τ is a rollout trajectory, r task ∈ {0, 1} is a terminal success indicator, bt is a segmentboundary flag, Φt is a shaping potential whose telescoping difference drives progress with discount γ ∈ (0, 1], r sub t is a sub-goal reaching indicator, r sm t penalizes consecutive-action change, r gr t aligns gripper state with the active primitive, and λseg, λprog, λsub, λsm, λgr > 0 are mixing weights.

Visual Grounding Failure is noted as the most common error type, where NS-VLA correctly identifies the current primitive but fails to spatially attend to the correct target object under strong environmental perturbations. p. 3
Where the paper says it

, u(M) , u(m) = op(m) , arg(m) , (4) where op(m) ∈ U identifies the symbolic operation occupying slot m and arg(m) ∈ Sarg encodes the manipulated object with the target referent. Outputs shorter than the cap Mmax are right-padded with a primitive that downstream pointer updates ignore, so the plan stays fixed across the episode.

Execution Instability occurs when generated continuous action sequences result in physical failure, often stemming from mapping sparse token features to high-precision continuous control.

Limitations and Future Work

Limitations include the fixed primitive vocabulary U, meaning instructions outside this set cannot be parsed by the rule-based extractor and silently fall back to the original instruction. The reward weights (λseg, λprog) in Eq. p. 4

Where the paper says it

Granularity-Matched Reward Streams. The high-level factor selects a primitive per decision and is coupled to a sparse milestone return firing at task completion and primitive transitions, while the low-level factor emits a chunk per decision and is coupled to a dense per-chunk shaping return: R h (τ ) = r task(τ ) + λsegX t bt, r l t = λprogγ Φt+1 − Φt + λsub r sub t + λsm r sm t + λgr r gr t , (8) where τ is a rollout trajectory, r task ∈ {0, 1} is a terminal success indicator, bt is a segmentboundary flag, Φt is a shaping potential whose telescoping difference drives progress with discount γ ∈ (0, 1], r sub t is a sub-goal reaching indicator, r sm t penalizes consecutive-action change, r gr t aligns gripper state with the active primitive, and λseg, λprog, λsub, λsm, λgr > 0 are mixing weights.

(8) are fixed across regimes, and the hierarchical policy is trained for 50 joint iterations with G=8 rollouts under a Qwen3-VL-2B encoder backbone. p. 5
Where the paper says it

The symbolic encoder is paired with the Qwen3-VL-2B backbone. Hierarchical RL: G=8 rollouts per group, βKL=0.

Future work suggests replacing the rule-based extractor and fixed primitive set U with an unsupervised primitive discovery module. p. 3
Where the paper says it

, M} is the admissible index set enforcing a forward step at most one slot, softmax(·)u(k) reads the logit at slot k, and the indicator 1[·] zeroes out non-admissible slots. The pointer is advanced via mt = arg maxk π h ϕ (k | ψ¯ t, mt−1), the executed primitive is uˆt = u (mt) , and the boundary flag bt = 1[mt ̸= mt−1] is emitted for reward shaping.

Additionally, treating the reward weights (λseg, λprog) as learnable parameters co-optimized inside the joint loss of g4. p. 4
Where the paper says it

Granularity-Matched Reward Streams. The high-level factor selects a primitive per decision and is coupled to a sparse milestone return firing at task completion and primitive transitions, while the low-level factor emits a chunk per decision and is coupled to a dense per-chunk shaping return: R h (τ ) = r task(τ ) + λsegX t bt, r l t = λprogγ Φt+1 − Φt + λsub r sub t + λsm r sm t + λgr r gr t , (8) where τ is a rollout trajectory, r task ∈ {0, 1} is a terminal success indicator, bt is a segmentboundary flag, Φt is a shaping potential whose telescoping difference drives progress with discount γ ∈ (0, 1], r sub t is a sub-goal reaching indicator, r sm t penalizes consecutive-action change, r gr t aligns gripper state with the active primitive, and λseg, λprog, λsub, λsm, λgr > 0 are mixing weights.

3 would let the shaping signal adapt to each task category’s trajectory length. p. 4
Where the paper says it

Granularity-Matched Reward Streams. The high-level factor selects a primitive per decision and is coupled to a sparse milestone return firing at task completion and primitive transitions, while the low-level factor emits a chunk per decision and is coupled to a dense per-chunk shaping return: R h (τ ) = r task(τ ) + λsegX t bt, r l t = λprogγ Φt+1 − Φt + λsub r sub t + λsm r sm t + λgr r gr t , (8) where τ is a rollout trajectory, r task ∈ {0, 1} is a terminal success indicator, bt is a segmentboundary flag, Φt is a shaping potential whose telescoping difference drives progress with discount γ ∈ (0, 1], r sub t is a sub-goal reaching indicator, r sm t penalizes consecutive-action change, r gr t aligns gripper state with the active primitive, and λseg, λprog, λsub, λsm, λgr > 0 are mixing weights.

Augmenting the joint update of Eq. p. 4
Where the paper says it

Proposition 3. The joint update preserves optimality and KL-bounds the high-level policy drift.

(9) with a process-supervision term would let πΘ trade success rate against deployment cost. p. 1
Where the paper says it

(ii) Backbone-bound generalization. All generalization passes through the pretrained backbone, so unseen instructions collapse the success rate [14] while acquiring a new task stays tied to pretraining cost.

The framework is bound to a single myCobot 320pi platform with a wrist plus third-person Realsense D435i pair for real-world deployment. p. 8
Where the paper says it

As shown in Figure 10, the workspace combines a wrist-mounted Realsense D435i camera with a third-person Realsense D435i camera. Tasks fall into three categories: three Pick tasks for single-object grasping, three Move tasks for pose-shifting, and four Place tasks for placement and stacking.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Neuro-Symbolic Encoder for Plan-Constrained Primitive Inference improves instruction grounding by parsing language into a plan of primitives and tracking execution with a monotone pointer, exposing compositional structure as a backbone-agnostic inductive bias. This ensures that the model learns reusable atomic operations rather than relying solely on monolithic backbones.

  2. Neuro-Symbolic Solver provides primitiveconditioned input for a chunked action backbone, bridging the symbolic decision to continuous control. This allows the VLA model to condition its low-level actions on a specific, relevant primitive, augmenting rather than replacing the underlying VLA.

  3. Hierarchical Joint Policy Optimization uses rewardgranularity matching by coupling high-level factors with sparse milestone rewards and low-level factors with dense perstep rewards. This matches the reward granularity to the policy type, leading to more effective learning for both primitive selection and continuous control.

  4. The system exhibits superior zero-shot generalizability because it learns reusable primitives from structured plans, enabling generalization to unseen tasks by leveraging this compositional structure instead of being restricted by a backbone-bound generalization.

  5. The model demonstrates expanded exploration space through Hierarchical Joint Policy Optimization, which updates the high-level factor with sparse milestone rewards and the low-level factor with dense perstep rewards. This leads to more effective online reinforcement learning that avoids collapsing onto a single trajectory distribution.

  6. The framework maintains optimal policies by using progress term r prog t = γΦt+1−Φt in Eq. (8), which matches the canonical potential-based shaping increment, ensuring that the dense per-chunk shaping does not pull the optimum off the manifold defined by the unshaped objective.

  7. The model can be used as a drop-in module across different backbones, as shown by all three lines slope upward and the relative ordering of the backbones is preserved before and after wrapping, supporting its role as a backbone-agnostic compositional interface.

  8. The system enables real-world deployment on hardware like the myCobot 320pi to achieve high success rates across pick, move, and place tasks, showing gains over simpler models by utilizing structured planning for physical manipulation.

Abstract

Vision-Language-Action (VLA) models ground natural-language instructions in visual observations to generate actions for robotic manipulation. On long-horizon tasks, existing VLA models face structure-blind backbones, backbone-bound generalization, and flat single-objective optimization. To address these challenges, we propose a novel Neuro-Symbolic Vision-Language-Action (NS-VLA) framework. It introduces a Neuro-Symbolic Encoder that parses the instruction into a primitive plan tracked by a gated monotone stage pointer, a Neuro-Symbolic Solver that conditions a chunked VLA policy on the active sub-instruction, and Hierarchical Joint Policy Optimization, which fits the pointer on demonstration segments and refines the action policy with success-filtered online learning. Even when trained on few demonstrations, NS-VLA clearly leads on LIBERO and the held-out LIBERO-Plus (76.5% and 51.8% against 71.2% and 45.4% for its backbone), exceeds the strongest baseline by 12.7 points on long-horizon tasks, and raises real-robot success from 40.0% to 60.0%. Our code is publicly available at https://github.com/Zuzuzzy/NS-VLA.

Sources

Related papers