PPTBench: Can Coding Agents Reconstruct the Visual World through Structured, Editable Slides
cs.CL, cs.AI
Submitted: 2026-08-31
Updated: 2026-09-28
Project page: https://lab.einsia.ai/pptbench
License: http://creativecommons.org/licenses/by/4.0/
The gist: Coding agents are increasingly moving beyond text-based software tasks to reconstruct visual targets through code.
Terminology
Abstract
Coding agents are increasingly moving beyond text-based software tasks to reconstruct visual targets through code. This capability, visual coding, requires agents to translate their understanding of visual targets into executable code. Slides provide a natural testbed for this capability, combining rich visual structure with objects that can be programmatically created and edited. To measure this capability, we introduce PPTBench, a benchmark for reconstructing scientific flow diagrams as editable PowerPoint slides. PPTBench covers 500 scientific flow-diagram tasks across 10 presentation domains, drawn from real research papers, and is evaluated with a four-stage agentic judge covering artifact validity, process and connector fidelity, rendering quality, and visual fidelity. Evaluation of ten models across 36 model--harness--effort configurations shows a substantial gap in reliable visual coding: the best configuration achieves 77.34, while the median across configurations is 24.38. Fine-grained analysis shows that agents can generally produce valid slide files, but struggle to produce high-quality reconstructions that faithfully recover the semantics and visual structure of the target. Further analysis shows that increasing reasoning effort primarily improves hard-gate passage rather than mean detail quality on each configuration's passing tasks, while configurations with more inspection tend to achieve higher overall scores. PPTBench establishes a measurable testbed for studying visual coding and advancing agents toward more reliable visual creation.
Sources
- AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZ
- DeTikZify: Synthesizing Graphics Programs for Scientific Figures and Sketches with TikZ
- MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark
- PresentBench: A Fine-Grained Rubric-Based Benchmark for Slide Generation
- MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?
- Frontier-Eng: Benchmarking Self-Evolving Agents on Real-World Engineering Tasks with Generative Optimization
- Design First, Code Later: Aesthetically Pleasing Template-Free Slides Generation
- AutoPresent: Designing Structured Visuals from Scratch
- PPTC Benchmark: Evaluating Large Language Models for PowerPoint Task Completion
- Paper2SysArch: Structure-Constrained System Architecture Generation from Scientific Papers
- SciFig: Towards Automating Editable Figure Generation for Scientific Papers
- COLE: A Hierarchical Generation Framework for Multi-Layered and Editable Graphic Design
- FigureQA: An Annotated Figure Dataset for Visual Reasoning
- Prometheus: Inducing Fine-grained Evaluation Capability in Language Models
- Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Dataset
- VL-RewardBench: A Challenging Benchmark for Vision-Language Generative Reward Models
- DiagramEval: Evaluating LLM-Generated Diagrams via Graphs
- Paper2Poster: Towards Multimodal Poster Automation from Scientific Papers
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering