TikZilla: Scaling Text-to-TikZ with High-Quality Data and Reinforcement Learning
cs.AI, cs.CL, cs.CV
Submitted: 2026-03-03
Updated: 2026-09-01
Comments: 33 pages, 18 figures, and 13 tables. Models and datasets: https://huggingface.co/collections/nllg/tikzilla . Code: https://github.com/NL2G/TikZilla
Journal ref: Proceedings of the Fourteenth International Conference on Learning Representations (ICLR 2026), 2026
License: http://creativecommons.org/licenses/by-sa/4.0/
The gist: Large language models (LLMs) are increasingly used to assist scientists across diverse workflows.
Terminology
Abstract
Large language models (LLMs) are increasingly used to assist scientists across diverse workflows. A key challenge is generating high-quality figures from textual descriptions, often represented as TikZ programs that can be rendered as scientific images. Prior research has proposed a variety of datasets and modeling approaches for this task. However, existing datasets for Text-to-TikZ are too small and noisy to capture the complexity of TikZ, causing mismatches between text and rendered figures. Moreover, prior approaches rely solely on supervised fine-tuning (SFT), which does not expose the model to the rendered semantics of the figure, often resulting in errors such as looping, irrelevant content, and incorrect spatial relations. To address these issues, we construct DaTikZ-V4, a dataset more than four times larger and substantially higher in quality than DaTikZ-V3, enriched with LLM-generated figure descriptions. Using this dataset, we train TikZilla, a family of small open-source Qwen models (3B and 8B) with a two-stage pipeline of SFT followed by reinforcement learning (RL). For RL, we leverage an image encoder trained via inverse graphics to provide semantically faithful reward signals. Extensive human evaluations with over 1,000 judgments show that TikZilla improves by 1.5-2 points over its base models on a 5-point scale, surpasses GPT-4o by 0.5 points, and matches GPT-5 in the image-based evaluation, while operating at much smaller model sizes.
Sources
- Qwen2.5-VL Technical Report
- TikZero: Zero-Shot Text-Guided Graphics Program Synthesis
- PaliGemma: A versatile 3B VLM for transfer
- Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation
- RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning
- Accelerating scientific discovery with Co-Scientist
- The Llama 3 Herd of Models
- LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models
- Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models
- Tulu 3: Pushing Frontiers in Open Language Model Post-Training
- Decoupled Weight Decay Regularization
- GPT-4 Technical Report
- GPT-4o System Card
- Qwen2.5 Technical Report
- FigGen: Text to Scientific Figure Generation
- StarVector: Generating Scalable Vector Graphics Code from Images and Text
- Rendering-Aware Reinforcement Learning for Vector Graphics Generation
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Multimodal Large Language Models: A Survey
- Qwen3 Technical Report
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems