Animation2Code: Evaluating Temporal Visual Reasoning in Video-to-Code Generation
cs.CV, cs.AI, cs.CL
Submitted: 2026-06-26
Updated: 2026-11-08
Code: https://github.com/alexey-pelykh/puppeteer-capture
License: http://creativecommons.org/licenses/by/4.0/
The gist: While recent vision-language models (VLMs) have achieved significant improvements on static visual-to-code tasks such as generating code for webpages, charts, or SVGs…
Terminology
Abstract
While recent vision-language models (VLMs) have achieved significant improvements on static visual-to-code tasks such as generating code for webpages, charts, or SVGs, it remains unclear whether they can recover temporal dynamics when motion is present. To this end, we introduce Animation2Code, a benchmark for evaluating temporal visual reasoning via reconstructing executable web animation code from videos. Animation2Code consists of 1,069 web animation videos with diverse visual appearances and motion patterns, paired with corresponding HTML/CSS/JavaScript implementations. We propose two human-aligned metrics, appearance similarity and temporal similarity, which allow us to disentangle visual fidelity from temporal alignment when comparing rendered animations against ground-truth samples. Benchmarking state-of-the-art VLMs on this dataset shows that current VLMs struggle to maintain temporal consistency in reconstruction, even when achieving high appearance similarity, including under finetuning and iterative refinement settings. Code and data are available at https://anya-ji.github.io/animation2code-website.
Sources
- GPT-4 Technical Report
- Dynamic Dynamic Time Warping
- TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Models
- WebVR: Benchmarking Multimodal LLMs for WebPage Recreation from Videos via Human-Aligned Visual Rubrics
- DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data
- Liger Kernel: Efficient Triton Kernels for LLM Training
- Learning Transferable Temporal Primitives for Video Reasoning via Synthetic Videos
- Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Dataset
- TimeBlind: A Spatio-Temporal Compositionality Benchmark for Video LLMs
- VideoVista: A Versatile Benchmark for Video Understanding and Reasoning
- Decomate: Leveraging Generative Models for Co-Creative SVG Animation
- AnyAni: An Interactive System with Generative AI for Animation Effect Creation and Code Understanding in Web Development
- Improved Iterative Refinement for Chart-to-Code Generation via Structured Instruction
- SlidesGen-Bench: Evaluating Slides Generation via Computational and Quantitative Metrics
- CLEVRER: CoLlision Events for Video REpresentation and Reasoning
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs