VectorGym: A Multi-Task Benchmark for SVG Code Generation, Sketching and Editing
cs.GR, cs.AI, cs.CV
Submitted: 2026-02-22
Updated: 2026-08-31
Comments: Accepted to EMNLP 2026. Camera-ready version. Updated author list and author order
License: http://creativecommons.org/licenses/by/4.0/
The gist: We introduce VectorGym, a comprehensive benchmark suite for Scalable Vector Graphics (SVG) that spans generation from text and sketches, complex editing, and visual understanding.
Terminology
Abstract
We introduce VectorGym, a comprehensive benchmark suite for Scalable Vector Graphics (SVG) that spans generation from text and sketches, complex editing, and visual understanding. VectorGym addresses the lack of realistic, challenging benchmarks aligned with professional design workflows. Our benchmark comprises four tasks with expert human-authored annotations: the novel Sketch2SVG task (VG-Sketch); a new SVG editing dataset (VG-Edit) featuring complex, multi-step edits with higher-order primitives; Text2SVG generation (VG-Text); and SVG captioning (VG-Cap). Unlike prior benchmarks that rely on synthetic edits, VectorGym provides gold-standard human annotations that require semantic understanding and design intent. We also provide a multi-task reinforcement learning baseline that jointly optimizes across all four tasks using rendering-based rewards. This baseline, built on GRPO with curriculum learning, trains a Qwen3-VL 8B model that achieves state-of-the-art performance among open-source models, surpassing much larger models including Qwen3-VL 235B and matching GPT-4o. We also introduce a VLM-as-a-Judge metric for SVG generation, validated through human correlation studies. Our evaluation of frontier VLMs reveals significant performance gaps, positioning VectorGym as a rigorous framework for advancing visual code generation.
Sources
- FIGR: Few-shot Image Generation with Reptile
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- The Stack: 3 TB of permissively licensed source code
- UniSVG: A Unified Dataset for Vector Graphic Understanding and Generation with Multimodal Large Language Models
- Leveraging Large Language Models for Scalable Vector Graphics-Driven Image Understanding
- Part I: Tricks or Traps? A Deep Dive into RL for LLM Reasoning
- SVGEditBench V2: A Benchmark for Instruction-based SVG Editing
- IconShop: Text-Guided Vector Icon Synthesis with Autoregressive Transformers
- DINOv2: Learning Robust Visual Features without Supervision
- OmniSVG: A Unified Scalable Vector Graphics Generation Model
- FigGen: Text to Scientific Figure Generation
- VGBench: Evaluating Large Language Models on Vector Graphics Understanding and Generation
- Rendering-Aware Reinforcement Learning for Vector Graphics Generation
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
Related papers
- SCRIPT: Scalable Diffusion Policy with Multi-stage Training for Language-driven Physics-Based Humanoid Control
- CADReasoner: Iterative Program Editing for CAD Reverse Engineering
- QuadLink: Autoregressive Quad-Dominant Mesh Generation via Point-Relation Learning
- DrawVideo: Grounded and Faithful Multi-Shot Video Generation from Storyboard Keyframe Sketches
- MotionPersona: Real-Time Locomotion Control across Personas, Bodies, and Styles
- GaussianBench: Physics-Fidelity Evaluation for Gaussian Scene Representations