FoldingAgent: Inferring Parametric Origami Procedures from Demonstration Videos
cs.CV, cs.AI, cs.GR
Submitted: 2026-08-31
Updated: 2026-08-31
Comments: Project Page: https://maya-moriya.github.io/origami-page/ Accepted to SIGGRAPH ASIA 2026
Code: https://github.com/edemaine/fold
Project page: https://maya-moriya.github.io/origami-page
License: http://creativecommons.org/licenses/by/4.0/
The gist: We present FoldingAgent, an agentic framework for inferring explicit parametric folding programs directly from origami demonstration videos.
Terminology
Abstract
We present FoldingAgent, an agentic framework for inferring explicit parametric folding programs directly from origami demonstration videos. Our framework leverages the reasoning power of a pre-trained Vision-Language Model (VLM) equipped with a suite of specialized tools that enable the agent to simulate geometric transitions, verify physical plausibility, retrieve and compare visual content, and evaluate its own predictions. To translate visual content into folding programs, we define a parametric space that consists of the paper's geometry and a set of parametric folding actions. Unlike models that predict static crease patterns, our agent operates sequentially and possesses the ability to re-plan its actions, effectively mitigating the compounding errors inherent in multi-step folding. Our approach takes a step toward closing the gap between human origami knowledge, which is primarily shared through unstructured visual demonstrations, and computational methods, which typically rely on structured, parametric representations such as a crease pattern or an executable parametric plan. We evaluate our approach on PurelandFold, a newly curated benchmark of diverse Pureland origami videos with ground-truth geometry and action labels. Our results demonstrate that by combining VLM reasoning with a set of specialized tools and physical simulation, we can successfully transform unstructured visual demonstrations into executable, physically plausible folding procedures.
Sources
- Can AI Understand the Language of Origami?
- Vision-Language Models as Differentiable Semantic and Spatial Rewards for Text-to-3D Generation
- Sparks of Artificial General Intelligence: Early experiments with GPT-4
- Know3D: Prompting 3D Generation with Knowledge from Vision-Language Models
- ChartLlama: A Multimodal LLM for Chart Understanding and Generation
- Learn2Fold: Structured Origami Generation with World Model Planning
- IR3D-Bench: Evaluating Vision-Language Model Scene Understanding as Agentic Inverse Rendering
- LL3M: Large Language 3D Modelers
- Gen3DEval: Using vLLMs for Automatic Evaluation of Generated 3D Objects
- GamiBench: Evaluating Spatial Reasoning and 2D-to-3D Planning Capabilities of MLLMs with Origami Folding Tasks
- SketchAgent: Language-Driven Sequential Sketch Generation
- GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation
- ReAct: Synergizing Reasoning and Acting in Language Models
- Vision-as-Inverse-Graphics Agent via Interleaved Multimodal Reasoning
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models