Beyond Legibility: Benchmarking Visual Text Rendering and In-Place Editing in Unified Video Generation
cs.CV
Submitted: 2026-09-29
Updated: 2026-09-30
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: A video can exhibit convincing motion and photorealism yet fail immediately when visual text collapses.
Terminology
Abstract
A video can exhibit convincing motion and photorealism yet fail immediately when visual text collapses. Unlike generic scene content, visual text is unforgiving in video generation: minor stroke corruption, temporal instability, or editing errors instantly break legibility and realism. Existing benchmarks overlook this challenge by treating text as incidental or using static OCR metrics that ignore temporal dynamics. We introduce VidScribe, a unified diagnostic benchmark spanning four generation regimes: writing from language (T2V), transferring text identity from a reference (R2V), sustaining text under dynamics (I2V), and localized text editing (V2V). VidScribe contains 803 human-verified samples across a 12-axis conditionally orthogonal factor space covering Intrinsic Text Properties, Physical Imaging Conditions, and Temporal Behavior. For reliable evaluation, we build a track-grounded, gated suite with 11 shared metrics and 2 task-specific probes under strict measurability conditions. Benchmarking 11 commercial and open-source systems shows that video text capability is non-monolithic, with content recognition decoupled from stroke-level glyph correctness. Performance is highly task-asymmetric: I2V sustains text most reliably, whereas V2V editing is the primary bottleneck. Counter-intuitively, degradation concentrates on a small subset of text-centric structural and temporal factors rather than adverse imaging conditions. Further probes show that visual references improve glyph and typographic fidelity rather than content accuracy, while localized editing fails to isolate target text without corrupting undeclared source text. Beyond evaluation, VidScribe also provides an actionable training signal, where benchmark-aligned preference optimization measurably improves visual text generation. https://huggingface.co/datasets/Vicky0720/VidScribe.
Sources
- Parameter-Efficient Fine-Tuning of DINOv2 for Large-Scale Font Classification
- LTX-2: Efficient Joint Audio-Visual Foundation Model
- LoRA: Low-Rank Adaptation of Large Language Models
- Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidance
- InnoText: A Unified Model for Visual Text Generation and Editing
- UnrealText: Synthesizing Realistic Scene Text Images from the Unreal World
- DINOv2: Learning Robust Visual Features without Supervision
- Visual Text Processing: A Comprehensive Review and Unified Evaluation
- Gemini: A Family of Highly Capable Multimodal Models
- Wan: Open and Advanced Large-Scale Video Generative Models
- MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generation
- HunyuanVideo 1.5 Technical Report
- JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion
- SteerVTE: Seamless Video Text Editing with Style and Glyph Control
- Beyond Coherence: Benchmarking Professional Editing-Technique Execution in Multi-Shot Audio-Video Generation
- VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness
- TextPecker: Rewarding Structural Anomaly Quantification for Enhancing Visual Text Rendering
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models