LLM-Flax: LLM-Generated Rules and Object Guidance for Neuro-Symbolic Task Planning

arXiv:2604.26569 · cs.RO · Submitted 2026-04-29 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "LLM-Flax: LLM-Generated Rules and Object Guidance for Neuro-Symbolic Task Planning".

Paper summary: Rosa: So we're looking at this paper called LLM-Flax, and it claims to automate the whole process of setting up a neuro-symbolic task planner on a brand new domain. It says that instead of needing a human expert to write all those relaxation and complementary rules, and then solving hundreds of training problems for a graph neural network object scorer, you just give the LLM a PDDL domain file.

Dev: Right, so the core idea is this three-stage framework that aims to eliminate all those manual engineering steps by using a locally hosted LLM. Stage one automatically generates those relaxation and complementary rules through structured prompting with some validation and self-correction built in.

Taro: And then stage three is pretty ambitious—it replaces that whole domain-trained graph neural network object scorer entirely with zero-shot LLM object importance scoring, meaning it requires no training data at all.

Rosa: That sounds like a big shift because the manual effort you mentioned could take hours from an expert to just a single PDDL file. The paper is looking at how this system performs across different grid sizes, specifically ten by ten twelve by twelve and fifteen by fifteen grids in the MazeNamo benchmark <ref:2604.26569#pg1>.

Dev: And they report that the full system achieves an average success rate of zero point nine four five versus a manual baseline of zero point eight two eight, which is an improvement of one point seventeen seven. That suggests it’s doing better than just using those hand-crafted rules we usually have to write ourselves.

Taro: But they also hit a wall at the larger scale, specifically on the fifteen by fifteen Hard grid where the manual planner fails completely with zero success rate <ref:2604.26569#pg1>. They attribute that collapse to a problem in context, saying that problems have over two hundred objects, but the LLM prompt is capped at eighty state facts.

Rosa: So they found something interesting: while it works well on smaller and medium problems, like those twelve by twelve grids where it solves twenty-two out of thirty problems in an average of five point two four seconds, it struggles when the problem gets too big <ref:2604.26569#pg1>.

Dev: And they’re calling that a context-window bottleneck at scale, which points to the main challenge for future work. The LLM prompt size is limited to eighty state facts, and that's not enough information when you have hundreds of objects in a complex scenario like on that fifteen by fifteen grid <ref:2604.26569#pg1>.

Taro: It makes sense from a failure mode standpoint; if the context window isn't big enough, the zero-shot scoring just doesn't have enough data to make an accurate importance decision for all those objects. How does that affect how gracefully the system recovers when it hits that context limit?

Paper summary: Rosa: That brings us to stage two, which is where they introduce LLM-guided failure recovery with a budget policy. This policy explicitly reserves the API latency cost before each LLM call to make sure that if the system needs to fall back to those domain-trained relaxation rules, it doesn't get starved of resources.

Dev: They implemented a feasibility check there too, meaning recovery is skipped if the remaining time before the Step two deadline is less than tLLM plus treplan minimum plus tstep two minimum, which they conservatively set to eleven seconds. That’s a safety net built into the loop so you don't get stuck waiting for an API call when time is running out.

Taro: From my side, I wonder if that budget policy is robust enough when the world misbehaves in a real-time scenario; does reserving that latency cost actually prevent a dangerous stall? If the LLM recovery call takes longer than expected, you're still capped at that eleven second limit before you have to decide whether to proceed or not.

Rosa: It seems like they are trying to manage the trade-off between letting the system try and failing gracefully versus cutting off the plan generation because of time constraints. The whole point is that this framework lets you deploy on a new domain with no manual engineering required for those three stages.

Dev: And looking at it from an engineering side, it means a PDDL domain file becomes the main input instead of needing hours of expert configuration to build the necessary rules and training data for a GNN scorer. That drastically cuts down the deployment cost on new domains.

Taro: I think what's really interesting is that they’ve managed to replace the need for hundreds of training problems with zero-shot LLM importance scoring in stage three, which is a huge contribution for cross-domain generalization. It proves you don't always need massive datasets when you have a capable LLM doing the scoring.

Rosa: So, to wrap up this overview of LLM-Flax: it’s a system that automates rule generation, includes budget-aware recovery, and uses zero-shot LLMs for scoring. The paper shows strong performance on smaller problems but highlights the context window limitation as a major hurdle for scaling to larger domains.

Dev: And the conclusion they draw is that open-source LLMs have enough comprehension of PDDL to replace those domain-specific engineering components for smaller and medium problems, opening a practical path toward neuro-symbolic planning that doesn't require constant manual effort.

Taro: The implication for me is that we need to keep pushing on how the system handles those large state spaces; if you can solve the context window issue, this framework could become much more applicable to complex, real-world robotics where things get messy.

Rosa: It sounds like the next step for this research is focusing on expanding that context understanding so that it doesn't just stop working when the problem gets too big. We’ll see how they tackle that in future work.

Conclusion: Rosa: So we're wrapping up on LLM-Flax, which is titled "LLM-Generated Rules and Object Guidance for Neuro-Symbolic Task Planning." It’s about taking a PDDL domain file and automatically building the entire planning setup without needing an expert to do all that manual rule writing.

Dev: Right, so it’s this three-stage framework where one stage creates those rules, another handles recovery if things go wrong, and the third replaces the old object scoring with zero-shot LLM importance guidance. It's all built using a locally hosted LLM.

Taro: The authors are showing how you can use an open-source LLM to do most of the heavy lifting for setting up a neuro-symbolic planner on new domains, which used to take hours of manual engineering work.

Rosa: Yeah, what it means is that deploying a planning system in a brand new area doesn't require hiring an expert just to configure the rules; you just need the domain file.

Dev: And the most significant part they highlight is how stage three replaces that whole trained graph neural network object scorer with zero-shot LLM scoring, which means no more training data needed for that part.

Taro: It’s a big step toward cross-domain generalization because you can use this method on problems you haven't seen before without having to train a model on specific examples of those problems.

Rosa: But the paper does show its limits, especially when the problem gets really big—the context window size becomes a real bottleneck for accurate object importance scoring.

Dev: That context window issue is definitely where we need to focus next; if the system can't handle bigger state spaces, it won't scale well in complex real-world scenarios.

Taro: Exactly, so while it’s great for smaller or medium problems right now, the challenge is making that zero-shot guidance robust enough when there are hundreds of objects involved.

Rosa: It shows that open-source LLMs have enough understanding of PDDL to replace those hard domain-specific engineering parts for smaller tasks.

Dev: So the implication here is a much lower barrier to entry for building these kinds of planning systems, moving away from needing massive amounts of task-specific training data.

Taro: The next thing we need to watch is how they plan to fix that context limitation so this framework can actually tackle those huge, messy problems in the real world.

Seongmin Kim, Daegyu Lee

Jeonbuk National University

cs.RO

Submitted: 2026-04-29

Updated: 2026-11-16

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 83/100

The gist: Deploying a neuro-symbolic task planner on a new domain today requires significant manual effort: a domain expert must author relaxation and complementary rules…

Key concepts

Stage 1: Rule Generation
This stage uses a local open-source LLM to automatically create necessary relaxation and complementary rules for the planner. It employs structured prompting with strict format validation and self-correction to ensure the generated rules are syntactically correct, saving domain experts from writing these rules manually.
Stage 2: Failure Recovery
This mechanism intervenes when a planning step times out by having an LLM identify which objects are likely causing the blockage. It uses a budget policy that reserves API latency cost before each call to ensure the system doesn't run out of time before attempting a fallback relaxation.
Stage 3: Zero-Shot Object Scoring
This replaces a trained Graph Neural Network with an LLM that scores object importance without any prior training data. The LLM receives the goal and state facts to determine which objects are most relevant, drastically reducing the need for domain-specific training.
LLM-Flax Framework
The overall system is a three-stage pipeline designed to deploy complex neuro-symbolic planners on entirely new domains using only a PDDL domain file. It integrates automated rule generation, intelligent failure recovery, and zero-shot scoring to minimize manual setup time from hours to just one domain file.

Terminology

Summary

Deploying a neuro-symbolic task planner on a new domain today requires significant manual effort: a domain expert must author relaxation and complementary rules, and hundreds of training problems must be solved to supervise a Graph Neural Network (GNN) object scorer <ref:2604. p. 0

Where the paper says it

1 LLM-Flax : Generalizable Robotic Task Planning via Neuro-Symbolic Approaches with Large Language Models Seongmin Kim1 , Daegyu Lee2∗ Abstract—Deploying a neuro-symbolic task planner on a new domain today requires significant manual effort: a domain expert must author relaxation and complementary rules, and hundreds of training problems must be solved to supervise a Graph Neural Network (GNN) object scorer. We propose LLM-Flax, a threestage framework that eliminates all three sources of manual effort using a locally hosted LLM given only a PDDL domain file.

26569#pg2> Stage 1 automatically generates relaxation and complementary rules via structured prompting with format validation and selfcorrection <ref:2604. p. 0
Where the paper says it

We propose LLM-Flax, a threestage framework that eliminates all three sources of manual effort using a locally hosted LLM given only a PDDL domain file. Stage 1 automatically generates relaxation and complementary rules via structured prompting with format validation and selfcorrection.

p. 0
Where the paper says it

We propose LLM-Flax, a threestage framework that eliminates all three sources of manual effort using a locally hosted LLM given only a PDDL domain file. Stage 1 automatically generates relaxation and complementary rules via structured prompting with format validation and selfcorrection.

26569#pg4> Stage 3 replaces the domain-trained GNN entirely with zero-shot LLM object importance scoring, requiring no training data <ref:2604. p. 0
Where the paper says it

Stage 2 introduces LLM-guided failure recovery with a feasibility-gated budget policy that explicitly reserves API latency cost before each LLM call, preventing the downstream relaxation fallback from being starved. Stage 3 replaces the domain-trained GNN entirely with zero-shot LLM object importance scoring, requiring no training data.

26569#pg5>

Framework Overview

LLM-Flax is a threestage framework that eliminates all three sources of manual effort using a locally hosted LLM given only a PDDL domain file <ref:2604.26569#pg2> Stage 1 automatically generates relaxation and complementary rules via structured prompting with format validation and selfcorrection <ref:2604.26569#pg4> Stage 2 introduces LLM-guided failure recovery with a feasibility-gated budget policy that explicitly reserves API latency cost before each LLM call, preventing the downstream relaxation fallback from being starved <ref:2604. p. 0

Where the paper says it

Stage 1 automatically generates relaxation and complementary rules via structured prompting with format validation and selfcorrection. Stage 2 introduces LLM-guided failure recovery with a feasibility-gated budget policy that explicitly reserves API latency cost before each LLM call, preventing the downstream relaxation fallback from being starved.

26569#pg2> Stage 3 replaces the domain-trained GNN entirely with zero-shot LLM object importance scoring, requiring no training data <ref:2604. p. 0
Where the paper says it

Stage 2 introduces LLM-guided failure recovery with a feasibility-gated budget policy that explicitly reserves API latency cost before each LLM call, preventing the downstream relaxation fallback from being starved. Stage 3 replaces the domain-trained GNN entirely with zero-shot LLM object importance scoring, requiring no training data.

26569#pg5>

Rule Generation and Validation

Stage 1 involves using a locally hosted open-source LLM served via Ollama to generate rules for relaxation and complementary rule types using a structured prompting strategy <ref:2604.26569#pg4> The prompt design includes the purpose, the exact JSON schema annotated with field semantics, an explicit emphasis that all index values must be integers, never strings, a correct worked example, and an explicit wrong example showing the most common failure mode <ref:2604. p. 2

Where the paper says it

The LLM is queried twice per domain—once for each rule type— using the following prompting strategy. Prompt design Each prompt includes: (1) a concise naturallanguage explanation of the rule type’s purpose; (2) the exact JSON schema annotated with field semantics; (3) explicit emphasis that all index values must be integers, never strings; (4) a correct worked example; and (5) an explicit “wrong example” showing the most common failure mode (using strings as index values).

26569#pg4> LLM outputs are validated by VALIDATERULES which checks if all required keys are present and if all index fields contain lists of integers <ref:2604. p. 2
Where the paper says it

The full PDDL domain text is included verbatim so the LLM can inspect all predicate signatures. Validation and self-correction LLM outputs are validated by VALIDATERULES, which checks: (i) all required keys are present; (ii) all index fields contain lists of integers; (iii) all predicate names exist in the domain (parsed from the PDDL :predicates block); and (iv) no duplicate rules appear.

26569#pg4> Post-processing steps include deduplication of rules with identical JSON content and predicate filtering to silently drop rules referencing predicates absent from the domain <ref:2604. p. 2
Where the paper says it

Post-processing After validation passes, two post-processing steps are applied: • Deduplication: rules with identical JSON content (modulo key name) are merged. • Predicate filtering: rules referencing predicates absent from the domain are silently dropped.

26569#pg4>

Failure Recovery Mechanism

Stage 2 introduces LLMRECOVERYGUIDANCE, inserted between a Step 1 timeout and the Step 2 relaxation fallback, where the LLM identifies which excluded objects are likely blocking the plan <ref:2604.26569#pg2> A budget policy is implemented to ensure that API latency cost is explicitly reserved before each LLM call to prevent downstream relaxation from being starved <ref:2604. p. 0

Where the paper says it

In this paper we present LLM-Flax, a three-stage framework realizing this vision: • Stage 1 — LLM Rule Generation: Structured prompting with format validation and self-correction generates relaxation and complementary rules directly from the PDDL domain file. • Stage 2 — LLM Failure Recovery: When GNN-guided planning times out, an LLM analyzes the current state and goal to identify missing objects; a budget-aware policy ensures that API latency cost is explicitly reserved before the call, preventing the downstream relaxation step from being starved.

26569#pg2> This policy includes a feasibility check where recovery is skipped if the remaining time before the Step 2 deadline is less than tLLM +treplan,min +tstep2,min (set conservatively to 11 s) <ref:2604. p. 2
Where the paper says it

Budget policy LLM API calls impose a latency cost (≈3–5 s) that must be accounted for in per-problem timeout budgets. We adopt a three-part policy (see Appendix A for derivation): • Feasibility check: skip recovery if the remaining time before the Step 2 deadline is < tLLM +treplan,min+tstep2,min (set conservatively to 11 s).

26569#pg4>

Zero-Shot Object Scoring Limitations

Stage 3 replaces the domain-trained GNN with LLMOBJECTGUIDANCE, a zero-shot scorer that receives the goal, the list of all objects with types, and up to 80 state facts <ref:2604.26569#pg5> Scores are cached so only one LLM call is made per planning problem <ref:2604. p. 3

Where the paper says it

On the first call for a given state, the LLM receives the goal, the list of all objects with types, and up to 80 state facts, and returns a JSON dictionary mapping each object name to a score in [0, 1]. Scores are cached so only one LLM call is made per planning problem.

26569#pg5> The context-window bottleneck at scale points to the primary open challenge for future work, as the LLM prompt is capped at 80 state facts for larger problems <ref:2604. p. 0
Where the paper says it

Stage 3 demonstrates feasibility (SR 0. 720 on 12×12 Hard with no training data) but faces a context-window bottleneck at scale, pointing to the primary open challenge for future work.

26569#pg2>

Performance and Generalization

LLM-Flax achieves average SR 0.945 versus the manual baseline’s 0. p. 0

Where the paper says it

LLM-Flax achieves average SR 0. 945 versus the manual baseline’s 0.

828 (+0.117), matching or outperforming manual rules on every one of the eight benchmarks <ref:2604. p. 0
Where the paper says it

945 versus the manual baseline’s 0. 117), matching or outperforming manual rules on every one of the eight benchmarks.

26569#pg2> The most striking improvements are at larger scale, where LLM-Flax solves 22 of 30 problems at 5. p. 3
Where the paper says it

On stacking-light tasks (10×10 Easy/Hard, 15×15 Medium), LLM-Flax ties Manual at equal SR; on stacking-moderate tasks (10×10 Medium, 12×12 Medium/Hard), it gains +0. The most striking improvements are at larger scale.

24 s average on 12×12 Expert, where the manual planner fails entirely (SR 0. p. 0
Where the paper says it

On 12×12 Expert, LLM-Flax attains SR 0. 733 where the manual planner fails entirely (SR 0.

000) <ref:2604. p. 0
Where the paper says it

PLOI [3] introduced GNN-based object filtering, reducing the grounded state space while preserving validity through iterative threshold relaxation. Subsequent work explored neuro-symbolic skills [11], relational transition models [12], predicate invention [13], and efficient abstract arXiv:2604.

26569#pg2> The collapse on 15×15 Hard traces to a scale limitation: problems have 240+ objects, but the LLM prompt is capped at 80 state facts, providing insufficient context for accurate zero-shot scoring <ref:2604. p. 6
Where the paper says it

979 Full LLM-Flax 0. 1 The collapse on 15×15 Hard traces to a scale limitation: problems have 240+ objects, but the LLM prompt is capped at 80 state facts, providing insufficient context for accurate zero-shot scoring.

26569#pg2>

Conclusion and Impact

The combined Full LLM-Flax system requires only a PDDL domain file to deploy on a new domain, reducing the cost of deploying on a new domain from hours of expert engineering to a single PDDL file <ref:2604.26569#pg2> The Stage 3 zero-shot GNN replacement—which eliminates the 200-problem training requirement—is the most impactful contribution for cross-domain generalization <ref:2604. p. 6

Where the paper says it

The manual baseline requires hours of engineering per domain; LLM-Flax requires only the PDDL file. Under this framing, a < 1% average SR gap on MazeNamo is a strong result, and the Stage 3 zero-shot GNN replacement—which eliminates the 200-problem training requirement—is the most impactful contribution for cross-domain generalization.

26569#pg2> Open-source LLMs possess sufficient PDDL comprehension to replace domain-specific engineering components for small-to-medium problems, opening a practical path toward truly domain-agnostic neuro-symbolic planning <ref:2604. p. 7
Where the paper says it

Extending zero-shot scoring to larger problems is the primary open challenge. Our results establish that open-source LLMs possess sufficient PDDL comprehension to replace domain-specific engineering components for small-to-medium problems, opening a practical path toward truly domain-agnostic neuro-symbolic planning.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Stage 1 LLM Rule Generation can automatically generate relaxation and complementary rules from a PDDL domain file using structured prompting with format validation and selfcorrection, eliminating manual labor for rule authoring.

  2. Stage 3 zero-shot LLM Object Scoring replaces the GNN entirely, requiring no training data by having an LLM score object importance from the PDDL state and goal without any training data, which is most impactful for cross-domain generalization.

  3. Stage 2 LLM Failure Recovery can be implemented with a feasibility-gated budget policy that explicitly reserves API latency cost before each LLM call, preventing the downstream relaxation fallback from being starved.

  4. The Full LLM-Flax system can deploy on a new domain using only a PDDL domain file, reducing the cost of deployment from hours of expert engineering to a single PDDL file.

Abstract

Adapting a neuro-symbolic task planner to a new domain can require hand-authored symbolic rules and solved training instances for a learned object scorer. This paper presents LLM-Flax, which applies large language models (LLMs) to offline rule generation, failure recovery, and object scoring within the Flax planning pipeline. Generated rules undergo validation and corrective feedback before reuse, while runtime guidance operates within a shared planning budget. Experiments on MazeNamo and two additional domains show that the evaluated rule configuration improves planning success over manually authored rules while retaining the trained object scorer. Across eight MazeNamo settings, the mean success rate increases by approximately three percentage points. Controlled comparisons distinguish the effects of rule content from the choice of generator. Runtime recovery and object scoring do not establish an overall performance advantage under the evaluated planning budgets. These findings demonstrate the value of reusable LLM-generated rules and identify information quality and inference efficiency as central challenges for runtime guidance.

Sources

Related papers