UltraDub: Towards Authentic Dubbing by Unifying Visually-Steered Flow Learning and Trajectory Guidance
cs.CV, cs.AI, cs.MM, cs.SD
Submitted: 2026-10-05
Updated: 2026-10-05
Code: https://github.com/Breakthrough/PySceneDetect
Project page: https://galaxycong.github.io/ultra
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Sources
- LRS3-TED: a large-scale dataset for visual speech recognition
- StyleDubber: Towards Multi-Scale Style Learning for Movie Dubbing
- CoSyncDiT: Cognitive Synchronous Diffusion Transformer for Movie Dubbing
- CFG-Zero*: Improved Classifier-Free Guidance for Flow Matching Models
- Classifier-Free Diffusion Guidance
- MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for Zero-Shot Speech Synthesis
- VoiceCraft-Dub: Automated Video Dubbing with Neural Codec Language Models
- CFG-Ctrl: Control-Based Classifier-Free Diffusion Guidance
- InstructDubber: Instruction-based Alignment for Zero-shot Movie Dubbing
- Prosody-Enhanced Acoustic Pre-training and Acoustic-Disentangled Prosody Adapting for Movie Dubbing
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models