AV-FlowEdit: Training-Free Prompt-Driven Audio-Visual Editing with Event-Aware Spectral Trajectory Preservation
cs.SD, cs.CV, eess.AS
Submitted: 2026-10-09
Updated: 2026-10-09
License: http://creativecommons.org/licenses/by/4.0/
The gist: Joint audio-video diffusion models generate synchronized content, but editing existing clips requires target changes without disrupting non-target structure or cross-modal correspondence.
Abstract
Joint audio-video diffusion models generate synchronized content, but editing existing clips requires target changes without disrupting non-target structure or cross-modal correspondence. We present AV-FlowEdit, a training-free and inversion-free framework for prompt-driven audio-video editing. Our key insight is that source--target flow differences provide both cross-modal editing directions and preservation cues. Under shared perturbations, they jointly update the video and audio branches, supporting visual-only, audio-only, and joint edits while retaining pretrained cross-modal interactions. To suppress drift accumulated during sampling, Cross-Modal Event-Aware Spectral Trajectory Preservation combines flow saliency, aligned audio activity, multiscale low-frequency displacement, and diffusion-stage scheduling. This adaptively preserves unsupported regions while allowing intended audio-visual events. AV-FlowEdit requires no model training, external masks, or optimization-based inversion. Experiments show strong prompt alignment and temporal consistency, together with competitive audio quality and cross-modal coherence.
Sources
- Omni2Sound: Towards Unified Video-Text-to-Audio Generation
- LTX-Video: Realtime Video Latent Diffusion
- LTX-2: Efficient Joint Audio-Visual Foundation Model
- VABench: A Comprehensive Benchmark for Audio-Video Generation
- FlowAlign: Trajectory-Regularized, Inversion-Free Flow-based Image Editing
- HunyuanVideo: A Systematic Framework For Large Video Generative Models
- AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Tasks
- Versatile Editing of Video Content, Actions, and Dynamics without Training
- FlowDirector: Training-Free Flow Steering for Precise Text-to-Video Editing
- SyncEdit: Rethinking Lip Synchronization as Editing with Audio-Driven Diffusion Models
- Video-P2P: Video Editing with Cross-attention Control
- EasyV2V: A High-quality Instruction-based Video Editing Framework
- Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversion
- SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations
- Wan: Open and Advanced Large-Scale Video Generative Models
- AV-DiT: Efficient Audio-Visual Diffusion Transformer for Joint Audio and Video Generation
- V2Edit: Versatile Video Diffusion Editor for Videos and 3D Scenes
Related papers
- Few-Shot Open-Set Audio Classification via Transductive Prototype Refinement and Class Logit Enhancement
- Spectral Masking and Interpolation Attack (SMIA): A Black-box Adversarial Attack against Voice Authentication and Anti-Spoofing Systems
- Remix the Timbre: Diffusion-Based Style Transfer Across Polyphonic Stems
- AVMeme Exam: A Multimodal Multilingual Multicultural Benchmark for LLMs' Contextual and Cultural Knowledge and Thinking
- SoundWeaver: Compositional Warm-Starting for Text-to-Audio Diffusion Serving
- WASIL: In-the-Wild Arabic Spoken Interactions with LLMs