LiveFoley: Dynamic Real-Time Video-to-Audio Generation for Immersive Interactive Worlds
cs.SD, cs.MM
Submitted: 2026-11-04
Updated: 2026-11-04
Terminology
Sources
- Omni2Sound: Towards Unified Video-Text-to-Audio Generation
- WorldSonus: Bringing Sound to Worlds
- Classifier-Free Diffusion Guidance
- HelixWorld: A Real-time Interactive Audio-Visual World Model
- Efficient Training of Audio Transformers with Patchout
- dots.tts Technical Report
- FoleyGen: Visually-Guided Audio Generation
- MiniCPM4: Ultra-Efficient LLMs on End Devices
- DINOv2: Learning Robust Visual Features without Supervision
- SoundReactor: Frame-level Online Video-to-Audio Generation
- HunyuanVideo-Foley: Multimodal Diffusion with Representation Alignment for High-Fidelity Foley Audio Generation
- Meta Audiobox Aesthetics: Unified Automatic Quality Assessment for Speech, Music, and Sound
- SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features
- Audio-Visual World Models: Learning Physically Grounded Multisensory Dynamics
- SpeakerVid-5M: A Large-Scale High-Quality Dataset for Audio-Visual Dyadic Interactive Human Generation
- VoxCPM: Tokenizer-Free TTS for Context-Aware Speech Generation and True-to-Life Voice Cloning
- VoxCPM2 Technical Report
Related papers
- Speech-DRAME: A Framework for Human-Aligned Benchmarks in Speech Role-Play
- MuLaCover: Flexible Cover Song Generation via Symbolic Tonal Control
- Optimising Neural Speech Codecs for 300bps Communication using Reinforcement Learning
- Few-Shot Open-Set Audio Classification via Transductive Prototype Refinement and Class Logit Enhancement
- Spectral Masking and Interpolation Attack (SMIA): A Black-box Adversarial Attack against Voice Authentication and Anti-Spoofing Systems
- Remix the Timbre: Diffusion-Based Style Transfer Across Polyphonic Stems