Less is More: Data-Efficient Adaptation for Controllable Text-to-Video Generation

arXiv:2511.17844 · cs.CV, cs.AI · Submitted 2025-11-21 · Read on arXiv

cs.CV, cs.AI

Submitted: 2025-11-21

Updated: 2026-04-08

Journal ref: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14811-14821 (2026)

Code: https://github.com/csh-apprentice/Less_Is_More

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

The gist: Fine-tuning large-scale text-to-video diffusion models to add new generative controls, such as those over physical camera parameters (e.g., shutter speed or aperture)…

Terminology

Abstract

Fine-tuning large-scale text-to-video diffusion models to add new generative controls, such as those over physical camera parameters (e.g., shutter speed or aperture), typically requires vast, high-fidelity datasets that are difficult to acquire. In this work, we propose a data-efficient fine-tuning strategy that learns these controls from sparse, low-quality synthetic data. We show that not only does fine-tuning on such simple data enable the desired controls, it actually yields superior results to models fine-tuned on photorealistic "real" data. Beyond demonstrating these results, we provide a framework that justifies this phenomenon both intuitively and quantitatively.

Sources

Related papers