VETR: Execution-Grounded Self-Improving Agent for Symbolic Graphics Reasoning
cs.AI
Submitted: 2026-10-09
Updated: 2026-10-12
Project page: https://osu-nlp-group.github.io/MagicBrush
License: http://creativecommons.org/licenses/by/4.0/
The gist: Visual agents for symbolic graphics reasoning depend on executable world representations, yet continued improvement remains bottlenecked by ambiguous supervision from final renders.
Abstract
Visual agents for symbolic graphics reasoning depend on executable world representations, yet continued improvement remains bottlenecked by ambiguous supervision from final renders. Recent agentic systems refine scenes through repeated interaction, but they lack mechanisms to recursively extract actionable evidence from execution traces, overlooking which proposed symbolic edits survived, disappeared, or interfered during program execution. To address this, we propose VETR (Verified Execution Trajectories for Refinement), a framework that enables agentic recursive self-improvement over executable world representations for symbolic graphics reasoning. Within each task, VETR retains aligned program/render records with bounded runtime observations and introduces an execution refiner that compiles observed writes and state changes into constrained alternative programs. The refiner validates their endpoints and preservation conditions against captured runtime state; the ordinary visual verifier then assesses goal relevance. Repair construction makes no additional language-model calls, but can require additional executions. Evaluations on BlenderGym (280 tasks) and BlenderBench (27 released tasks) with GPT-5.6-luna and Qwen3.8-27B show that VETR consistently outperforms existing refinement strategies, including BlenderAlchemy and VIGA. Controlled ablations evaluate the contributions of execution refinement and role-asymmetric evidence organization under fixed generator-interaction caps.
Sources
- HiLSVA: Design and Evaluation of a Human-in-the-Loop Agentic System for Scientific Visualization
- SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents
- LayoutGPT: Compositional Visual Planning and Generation with Large Language Models
- CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
- BlenderGym: Benchmarking Foundational Model Systems for Graphics Editing
- Thinking in Blender: Staged Executable Inverse Graphics with Vision-Language Models
- Prompt-to-Prompt Image Editing with Cross Attention Control
- Self-Refine: Iterative Refinement with Self-Feedback
- Toward AI VIS Co-Scientists: A General and End-to-End Agent Harness for Solving Complex Data Visualization Tasks
- Reflexion: Language Agents with Verbal Reinforcement Learning
- A-MEM: Agentic Memory for LLM Agents
- Holodeck: Language Guided Generation of 3D Embodied AI Environments
- Vision-as-Inverse-Graphics Agent via Interleaved Multimodal Reasoning
- Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems