IDfRA: Self-Verification for Iterative Design in Robotic Assembly

arXiv:2509.16998 · cs.RO · Submitted 2025-09-21 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: I'm Rosa.

Dev: And I'm Dev, here with Taro.

Rosa: Today's paper: "IDfRA: Self-Verification for Iterative Design in Robotic Assembly".

Dev: The gist: IDfRA, a framework using iterative cycles of planning, execution, verification, and re-planning informed by self-assessment,

Rosa: First, who's behind it and why it matters.

Paper summary: Rosa: So we’re talking about this paper called "IDfRA: Self-Verification for Iterative Design in Robotic Assembly." It sounds like they’re tackling the problem of designing things for robots to put together, which is Design for Robotic Assembly or DfRA.

Dev: Right. They're looking at how traditional ways of doing this are too manual and slow for complex objects nineteen <ref:2509.16998#pg1>.

Rosa: Yeah, and they’re saying that instead of just using big language models or heuristic strategies, IDfRA uses a loop where the system plans, executes, checks itself against the real world, and then replans if it fails.

Dev: It seems like the main claim is that this iterative process lets you improve design quality within an environment that isn't perfectly defined at first.

Taro: So they’re trying to get around needing a perfect initial map of everything before you start building, which is a big deal in real manufacturing settings twenty-one <ref:2509.16998#pg1>.

Rosa: Exactly. The core idea is using self-verification as the feedback mechanism. They argue that having the robot assess its own actions and outcomes is really vital when things get uncertain or safety-critical twenty <ref:2509.16998#pg1>.

Dev: That means if it messes up, it needs to know how to assess that mistake and correct itself properly for a real system.

Taro: I'm interested in how gracefully the system recovers when the world misbehaves, not just how often it fails twenty-three <ref:2509.16998#pg1>.

Rosa: That’s what they address. IDfRA aims to solve three main needs in modern DfRA: adaptability through iterative refinement, safety through that self-assessment, and efficiency by automating the whole thing from start to finish twenty-one <ref:2509.16998#pg1>.

Dev: So, the contribution of this work is presenting IDfRA itself as a framework for Iterative DfRA that combines self-verification with re-planning to enable this kind of adaptability twenty-one <ref:2509.16998#pg1>.

Rosa: And they also show how this system generates designs that are not just semantically meaningful but also physically feasible, and those designs improve through the feedback they get twenty-three <ref:2509.16998#pg1>.

Taro: That improvement aspect is interesting. Does that mean the design quality actually gets better every time it loops?

Dev: It does, although the paper notes that this improvement isn't always smooth or strictly increasing across every single step twenty-three <ref:2509.16998#pg1>.

Paper summary: Rosa: Well, it seems like they’ve built a system where you start with a minimal design constructed by a simulated robot, then you feed an image of that attempt into a VLM Judge module <ref:2509.16998#pg3>.

Dev: And based on that feedback and previous plans, the Replanner module generates an updated assembly design, which is then executed again by the simulated robot in this iterative cycle <ref:2509.16998#pg3>.

Taro: So they’re using a GIF of the attempt to guide the assessment process, which sounds like it's trying to mimic how a real system would evaluate its own progress <ref:2509.16998#pg3>.

Rosa: Right. Then after ten iterations, they have this Selector module that uses another GPT-4o instance to pick the final design from the ones that don't require unavailable blocks <ref:2509.16998#pg3>.

Dev: So the experimental evaluation shows IDfRA gets up to seventy-three point three three percent top-one accuracy in semantic recognisability <ref:2509.16998#pg3>.

Taro: Seventy-three percent sounds pretty solid for an object recognition task, especially when compared to the baseline they are testing against <ref:2509.16998#pg3>.

Rosa: And on top of that, the resulting assembly plans show a construction success rate of eighty-six point nine zero percent across those six representative assemblies <ref:2509.16998#pg3>.

Dev: That physical feasibility result is important because it shows that these designs actually work when you run them through the simulated robotic arm in the test environment <ref:2509.16998#pg3>.

Taro: So, what does this mean for someone who just listens to the show? It means we’re looking at a way to design things for robots that can adapt without needing perfect blueprints beforehand.

Rosa: That’s the big picture, Taro. It moves DfRA away from relying on rigid, hard-coded physics simulators and towards something that learns what works through interaction <ref:2509.16998#pg3>.

Dev: It also addresses the need for safety by embedding this self-assessment directly into the loop, which is a key concern for real-time control systems <ref:2509.16998#pg2>.

Taro: I wonder what happens when the world throws something completely unexpected at it, like a block that doesn't fit the expected constraints?

Rosa: That’s where they’re testing its robustness against those kinds of unforeseen issues in this iterative process <ref:2509.16998#pg2>.

Dev: So, to wrap up this summary of "IDfRA: Self-Verification for Iterative Design in Robotic Assembly," it’s a framework that uses iterative cycles of planning and execution informed by self-assessment to improve design quality in under-specified environments <ref:2509.16998#pg2>.

Paper summary: Rosa: It really boils down to using AI reasoning—the LLM for planning and the vision language model for checking the results—to create a continuous loop of refinement that builds better designs step by step.

Dev: And as we looked at the experimental results, they showed this approach can achieve seventy-three point three three percent top-one accuracy in semantic recognisability <ref:2509.16998#pg3>.

Taro: So, when you compare it to previous work like BloxNet, IDfRA seems to be better at recognizing the actual structure name even on more complex targets like a "robot" or a "Taj Mahal" <ref:2509.16998#pg3>.

Rosa: That’s what the human evaluation survey showed, too, where IDfRA designs won in eight out of fifteen comparisons against BloxNet <ref:2509.16998#pg3>.

Dev: It also has this eighty-six point six seven percent overall success rate for construction across those six structures tested <ref:2509.16998#pg2>.

Taro: So, the implication is that we might start seeing designs that are not just plausible in theory but actually physically buildable through this kind of self-correction mechanism.

Rosa: That’s what it suggests for manufacturing: a system that can handle uncertainty by constantly checking its own work and adjusting the plan online <ref:2509.16998#pg3>.

Dev: The limitation they mentioned is that the LLM-based Position submodule can still have some inconsistencies even with prompt engineering improvements <ref:2509.16998#pg2>.

Taro: That means we still have to watch out for those occasional errors in the coordinate generation part of the plan <ref:2509.16998#pg2>.

Rosa: And they also pointed out that the designs are constrained by how well you can represent targets using a limited set of blocks provided <ref:2509.16998#pg2>.

Dev: It’s clear that while IDfRA is showing promise as a proof of concept, it still has areas to improve for real deployment <ref:2509.16998#pg3>.

Taro: And they suggested future work could involve adding persistent memory, like a module for common mistakes to avoid or a skill library, to keep the progress more consistent across iterations <ref:2509.16998#pg2>.

Rosa: So, IDfRA is showing that integrating self-verification with context-aware adaptation really has strong potential for unstructured manufacturing scenarios <ref:2509.16998#pg2>.

Dev: It’s a framework that uses iterative cycles of planning, execution, verification, and re-planning, each informed by self-assessment to progressively enhance design quality within a fixed yet initially under-specified environment <ref:2509.16998#pg3>.

Taro: It’s about moving the intelligence into the loop itself rather than just having a separate planning step before execution <ref:2509.16998#pg2>.

Rosa: That’s what this paper is all about, IDfRA: Self-Verification for Iterative Design in Robotic Assembly.

Conclusion: Rosa: So we've been looking at IDfRA, this paper about Iterative Design for Robotic Assembly.

Dev: Yeah, it's about this closed loop where you plan, execute, check yourself against the real world using a vision language model, and then replan if something goes wrong.

Taro: It seems to be trying to get around needing a perfect initial map of everything before you start building.

Rosa: Exactly. The authors are arguing that this lets you improve design quality in an environment that isn't perfectly defined at the start.

Dev: They introduce this self-verification mechanism as crucial for safety, especially when things get uncertain in real systems.

Taro: I'm interested in how gracefully the system recovers when the world throws something completely unexpected at it during those iterations.

Rosa: So they show that by using this feedback loop, you can generate designs that are both semantically recognizable and physically feasible.

Dev: They report a construction success rate of eighty-six point six seven percent across six different assemblies tested in the simulation.

Taro: That physical feasibility result is important because it shows these designs actually work when you run them through the simulated robotic arm in the test environment.

Rosa: But they also have caveats about where this framework stops working, specifically that the LLM-based position submodule can still have some inconsistencies even with good prompting.

Dev: And they also note that the designs are constrained by how well you can represent targets using only a limited set of blocks provided in the system.

Taro: It sounds like future work could focus on adding persistent memory, maybe a module for common mistakes to avoid or a skill library to keep the progress more consistent across those iterations.

Rosa: So IDfRA is showing that integrating self-verification with context-aware adaptation can be really useful for unstructured manufacturing scenarios.

Dev: It's a framework that uses iterative cycles of planning and execution, each informed by self-assessment, to progressively enhance design quality in an environment where you don't have all the answers upfront.

University of Cambridge

cs.RO

Submitted: 2025-09-21

Updated: 2026-10-14

Comments: 8 pages, 8 figures. Published at ICRA 2026

Journal ref: 2026 IEEE International Conference on Robotics and Automation (ICRA), Vienna, Austria, 2026, pp. 1831-1838

DOI: 10.1109/ICRA57385.2026.11697014

Code: https://github.com/cam-cambridge/open-ended-assembly

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 77/100

The gist: The gist: IDfRA, a framework using iterative cycles of planning, execution, verification, and re-planning informed by self-assessment, progressively enhances design quality within an under-specified

Key concepts

IDfRA (Iterative Design for Robotic Assembly): A closed-loop framework combining an LLM for generating assembly plans, a VLM for assessing execution, and a re-planning step based on verification feedback. It allows the system to continuously refine its design through planning, execution, and self-assessment cycles.
VLM-Based Self-Verification Module (Judge): A module using a model like GPT-4o that analyzes a video of the robot's assembly attempt alongside the target structure and current plan. It infers the semantic roles of blocks to check for missing pieces, stability issues, and semantic resemblance to the goal.
LLM-Based Re-Planning Module (Replanner): A tiered system of GPT-4o instances that takes feedback from the Judge module. It generates new assembly plans, correcting illogical sequences and calculating explicit 3D coordinates for block placement to ensure a feasible bottom-up assembly.

Terminology

Verification — 29× in this paper · explained in 2 episodes
Iterative Refinement — 13× in this paper · explained in 3 episodes
Accuracy — 12× in this paper · explained in 1 episode
Adaptability — 9× in this paper · explained in 1 episode
Design for Robotic Assembly — 9× in this paper
Adaptation — 6× in this paper · explained in 1 episode
Task Planning — 6× in this paper · explained in 2 episodes
Core — 4× in this paper · explained in 2 episodes
Iterative Design for Robotic Assembly — 3× in this paper
Prompt Engineering — 3× in this paper · explained in 5 episodes
Assembly Sequence Planning — 2× in this paper
Fidelity — 2× in this paper · explained in 2 episodes
Foundation Model — 2× in this paper · explained in 3 episodes
International Conference on Robotics and Automation — 2× in this paper
Pairwise Comparisons — 2× in this paper · explained in 2 episodes
Robustness — 2× in this paper · explained in 5 episodes
Transformer — 2× in this paper · explained in 2 episodes
Vision-language models — 2× in this paper · explained in 2 episodes
World Model — 2× in this paper · explained in 7 episodes
Affordance — 1× in this paper · explained in 1 episode
Benchmark — 1× in this paper · explained in 2 episodes
Causal Discovery — 1× in this paper · explained in 3 episodes
Computer-Aided Design — 1× in this paper
Consistency — 1× in this paper · explained in 3 episodes
Decode — 1× in this paper · explained in 2 episodes
Generalization — 1× in this paper · explained in 13 episodes
Grounding — 1× in this paper · explained in 3 episodes
Interpretability — 1× in this paper · explained in 15 episodes
Large Language Models (LLMs) — 1× in this paper · explained in 8 episodes
Physical Constraints — 1× in this paper · explained in 7 episodes
Procedural Knowledge — 1× in this paper · explained in 3 episodes
Semantic Alignment — 1× in this paper · explained in 4 episodes
Semantic Understanding — 1× in this paper · explained in 2 episodes
Sim-to-Real Gap — 1× in this paper · explained in 6 episodes
Vision-Language Model (VLM) — 1× in this paper · explained in 3 episodes

Summary

The gist: IDfRA, a framework using iterative cycles of planning, execution, verification, and re-planning informed by self-assessment, progressively enhances design quality within an under-specified environment by eliminating physics simulation with the real world itself.

Introduction and Motivation

Design for Robotic Assembly (DfRA) is increasingly important as manufacturing shifts toward flexible, adaptive systems in dynamic environments <ref:2509.16998#pg2>. Traditional approaches to DfRA rely on manual planning which is time-consuming, expensive and potentially impractical for complex objects <ref:2509.16998#pg3>. Large language models (LLM) have shown proficiency in semantic interpretation and robotic task planning, stimulating interest in their application to the automation of DfRA <ref:2509.16998#pg3>. Existing methodologies typically rely on heuristic strategies and rigid, hard-coded physics simulators that may not translate into real-world assembly contexts <ref:2509.16998#pg3>. IDfRA presents a framework using iterative cycles of planning, execution, verification, and re-planning each informed by self-assessment to progressively enhance design quality within a fixed yet initially under-specified environment thereby eliminating the physics simulation with the real world itself <ref:2509.16998#pg3>.

IDfRA Framework and Core Concepts

IDfRA is introduced as a framework for Iterative DfRA which integrates self-verification and re-planning to enable adaptability through iterative refinement <ref:2509.16998#pg2>. The framework employs self-verification as its feedback mechanism where a robotic system’s ability to assess its own actions and outcomes is vital in uncertain or safety-critical environments <ref:2509.16998#pg2>. IDfRA addresses three core needs in modern DfRA: adaptability to dynamic contexts via iterative refinement, safety through embedded self-assessment, and efficiency via end-to-end automation <ref:2509.16998#pg2>. The contributions of this work include IDfRA which is the first fully automated framework combining VLM-based selfverification with LLM-based re-planning for iterative design improvement <ref:2509.16998#pg3> and a systematic evaluation showing that IDfRA generates semantically meaningful, physically feasible designs that evolve through feedback providing insights into foundation model reasoning within DfRA <ref:2509.16998#pg3>.

System Architecture and Components

The IDfRA system architecture details an iterative refinement process where the process begins with a single candidate design generated by BloxNet with physics-based stability correction disabled to allow refinement of potentially unstable configurations <ref:2509.16998#pg2>. The framework consists of four key components:

VLM-Based Self-Verification Module (Judge): The Judge, implemented as a GPT-4o instance, is queried with a GIF of the robot’s latest assembly attempt and a structured prompt incorporating the target structure name, available blocks, missing blocks, and the current JSON-formatted assembly plan <ref:2509.16998#pg2>. The Judge analyses the GIF and infers semantic roles of each block based on visual input to return a structured JSON assessment including identification of unavailable blocks and associated design features, evaluation of whether any missing blocks at intermediate stages may affect overall stability, assessment of the semantic resemblance to the target structure, and suggestions for design improvement <ref:2509.16998#pg2>.
LLM-Based Re-Planning Module (Replanner): The Replanner comprises three GPT-4o instances arranged in a tiered structure to creatively incorporate feedback and progressively increase precision to generate a feasible assembly plan with explicit coordinates <ref:2509.16998#pg2>. Internally, the Replanner comprises three submodules: Replan processes all inputs and incorporates Judge feedback to generate a highlevel plan from scratch, Order corrects illogical sequencing to produce a feasible bottomup assembly, and Position generates explicit 3D centre coordinates (X, Y, Z) and yaw angles <ref:2509.16998#pg2>.
Robot Assembly Execution Module: Generated assembly plans are executed using a simulated robotic arm in a tabletop environment where blocks requiring dimension switching have their roll and pitch calculated programmatically <ref:2509.16998#pg2>. The simulation serves as a proxy for a real robotic system, and GIFs of assembly attempts are intended to mirror real-world execution videos to establish proof of concept <ref:2509.16998#pg2>.
VLM-Based Selector Module: After ten iterations, images are rendered for each assembly by spawning blocks at the generated poses under gravity and rigid body dynamics <ref:2509.16998#pg2>. Plans containing any missing blocks are disqualified and the remaining undergo a pairwise knockout process where a GPT-4o instance selects the more stable and semantically accurate design <ref:2509.16998#pg2>.

Experimental Evaluation

The experiments evaluate IDfRA on three objectives: semantic recognisability, physical feasibility, and iterative improvement <ref:2509.16998#pg2>.

A) Human Evaluation Survey: Responses from 100 participants show that IDfRA was preferred in 8 out of 15 designs achieving a win rate of 54.2% (813/1,500 pairwise comparisons) <ref:2509.16998#pg3>. IDfRA outperformed BloxNet on more complex structures with multiple components such as “robot” and “Taj Mahal” while BloxNet performed better on simpler targets such as “ceiling fan” and “bridge” <ref:2509.16998#pg3>.

B) VLM Test for Design Recognisability: IDfRA consistently outperforms BloxNet in this evaluation obtaining up to 73.33% top-1 accuracy at N = 10 <ref:2509.16998#pg3>. While IDfRA designs generally received higher rankings, both methods exhibited a common limitation where structures with similar high-level shapes were frequently confused <ref:2509.16998#pg3>.

C) Robot Assembly Test for Feasibility: As shown in Table II, IDfRA’s designs are physically executable by the simulated robotic arm with three out of six structures successfully assembled in all 10 trials <ref:2509.16998#pg3>.

D) Iterative Improvement Evaluation: Fig 6 illustrates that designs remained the same or improved across iterations on average with a greater standard deviation in later iterations <ref:2509.16998#pg3>. The results confirm that designs remain the same or improve via iterative refinement despite fluctuations <ref:2509.16998#pg3>.

Conclusion and Limitations

Experimental results demonstrate that IDfRA generates semantically recognisable and feasible assembly plans and that iterative refinement can improve Design for Robotic Assembly (DfRA) <ref:2509.16998#pg2>. The approach shows promise as a proof of concept but several limitations remain, including the LLM-based Position submodule being prone to occasional inconsistencies despite promptengineering improvements <ref:2509.16998#pg2>. Generated designs depend to some extent on the block set provided and are constrained by how well targets can be represented by a limited set of blocks <ref:2509.16998#pg2>. The simulation environment was intentionally kept minimal to serve as a proxy for real robotic systems with recorded GIFs mimicking real-world execution videos used in the planning pipeline <ref:2509.16998#pg2>. Future work could address this by incorporating persistent memory such as a “common mistakes to avoid” module or skill library to encourage more consistent progress across iterations <ref:2509.16998#pg2>. Despite these limitations, IDfRA achieves Top-1 accuracy of up to 73.33% in VLM-based object recognisability and outperforms BloxNet in 8 out of 15 designs based on pairwise human evaluations indicating enhanced semantic alignment <ref:2509.16998#pg2>. The generated plans are also physically feasible with an overall success rate of 86.67% across six representative assemblies <ref:2509.16998#pg2>. Iterative evaluation further shows that designs often improve across successive refinements despite fluctuations <ref:2509.16998#pg2>.

References

[1] J. F. Engelberger and G. C. Devol, “Programmed article transfer,” Patent US2 988 237A, May 13, 1954, https://patents.google.com/patent/US2988237A/en.

[2] Y. Jiang, Z. Huang, B. Yang, and W. Yang, “A review of robotic assembly strategies for the full operation procedure: planning, execution and evaluation,” Robotics and Computer-Integrated Manufacturing, vol. 78, p. 102366, 2022.

[3] A. R. Das and M. Koskinopoulou, “Toward sustainable manufacturing: A review on innovations in robotic assembly and disassembly,” IEEE Access, vol. 13, pp. 100 149–100 166, June 2025.

[4] G. Boothroyd, Assembly Automation and Product Design, 2nd ed. CRC Press, 2005.

[5] R. W. Kennard and L. A. Stone, “Computer aided design of experiments,” Technometrics, vol. 11, no. 1, pp. 137–148, 1969.

[6] T.-C. Chang, R. A. Wysk, and H.-P. Wang, Computer-Aided Manufacturing.

[7] Z. Bi and X. Wang, Computer Aided Design and Manufacturing.

[8] S. Hagemann and R. Stark, “An optimal algorithm for the robotic assembly system design problem: An industrial case study,” CIRP Journal of Manufacturing Science and Technology, 09 2020.

[9] X. Zhu, D. Jha, D. Romeres, L. Sun, M. Tomizuka, and A.

Improvements for AI systems

Based on IDfRA, which to our knowledge, is the first fully automated framework combining VLM-based selfverification with LLM-based re-planning for iterative design improvement, this system can autonomously generate and refine complex robotic assembly plans by integrating visual feedback with iterative planning cycles.
The system can address adaptability to dynamic contexts via iterative refinement by using the VLM Judge module to assess assembly attempts based on visual input and returning structured JSON assessments, which then inform the LLM-Based Re-Planning Module (Replanner) to produce updated plans, thereby enabling context-aware design refinement.
The system can enhance safety in uncertain environments by employing self-verification – a robotic system’s ability to assess its own actions and outcomes, allowing it to improve performance by enabling iterative refinement of task plans in both robotics [21]–[23] and other domains [24].
The architecture can achieve high semantic fidelity by using the Judge module to infer the semantic role of each block (e.g., identifying a small block on a “roof” as a “chimney”) based solely on visual input, fostering an independent understanding and unbiased feedback.
The system can ensure physical feasibility by having the Replanner's submodules generate plans that are then executed by the Robot Assembly Execution Module, which calculates necessary rotations for dimension switching and uses low-gain position control to ensure slow, smooth movements.
The system can improve design quality through explicit constraint management by using prompt engineering in the Replanner to minimize overhangs and enhance stability by employing strategies such as removing ambiguity, adding negative examples, and providing detailed instructions that guide positioning based on adjacent blocks.
The final selection process can be optimized for robustness by implementing a pairwise knockout process where a GPT-4o instance selects the more stable and semantically accurate design, automatically favoring stability and semantic resemblance.
Future iterations can incorporate persistent memory, such as a ‘common mistakes to avoid’ module or skill library [24], [34], to encourage more consistent progress across iterations, mitigating the current risk that no satisfactory design emerges (Fig 7c).