YuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality

arXiv:2609.33757 · eess.AS, cs.LG, cs.SD · Submitted 2026-09-27 · Read on arXiv

eess.AS, cs.LG, cs.SD

Submitted: 2026-09-27

Updated: 2026-09-27

Code: https://github.com/multimodal-art-projection/YuE

Project page: https://map-yue2.github.io/ABSTRACT

Terminology

• Alignment — 71× in this paper · explained in 1 episode
• Accuracy — 28× in this paper · explained in 1 episode
• Benchmark — 24× in this paper · explained in 2 episodes
• Transformer — 18× in this paper · explained in 2 episodes
• Adaptation — 17× in this paper · explained in 1 episode
• Quantization — 12× in this paper · explained in 12 episodes
• Fine-Tuning — 11× in this paper · explained in 5 episodes
• Fidelity — 9× in this paper · explained in 2 episodes
• Embeddings — 8× in this paper · explained in 2 episodes
• Aggregation — 7× in this paper · explained in 1 episode
• Consistency — 6× in this paper · explained in 3 episodes
• Representation Learning — 6× in this paper · explained in 2 episodes
• Flow Matching — 5× in this paper · explained in 8 episodes
• NAR Mixture-of-Transformers — 5× in this paper
• Adam — 4× in this paper · explained in 2 episodes
• Clustering — 4× in this paper · explained in 2 episodes
• Cosine Similarity — 4× in this paper · explained in 3 episodes
• Tokenization — 4× in this paper · explained in 2 episodes
• AUROC — 3× in this paper · explained in 2 episodes
• Analyzer — 3× in this paper · explained in 1 episode
• Foundation Model — 3× in this paper · explained in 3 episodes
• Prompt Adherence — 3× in this paper · explained in 2 episodes
• Ablation — 2× in this paper · explained in 1 episode
• Foundation Models — 2× in this paper · explained in 7 episodes
• Alignment Preferences — 1× in this paper · explained in 1 episode
• Calibration — 1× in this paper · explained in 7 episodes
• Core — 1× in this paper · explained in 2 episodes
• Generative Modeling — 1× in this paper · explained in 3 episodes
• Grounding — 1× in this paper · explained in 3 episodes
• Jazz Audio-Aligned Harmony — 1× in this paper
• Regularization — 1× in this paper · explained in 3 episodes
• Self-Supervised Learning — 1× in this paper · explained in 8 episodes
• Unified Framework — 1× in this paper · explained in 5 episodes
• Variational Bayes — 1× in this paper · explained in 2 episodes

Sources

• MusicLM: Generating Music From Text
• ChordFormer: A Conformer-Based Architecture for Large-Vocabulary Audio Chord Recognition
• Discogs-VI: A Musical Version Identification Dataset Based on Public Editorial Metadata
• Seed-Music: A Unified Framework for High Quality and Controlled Music Generation
• SymPAC: Scalable Symbolic Music Generation With Prompts And Constraints
• Qwen2-Audio Technical Report
• Simple and Controllable Music Generation
• Pushing the Frontier of Full-Song Generation: Hierarchical Autoregressive Planning Meets Flow-Matching Rendering
• High Fidelity Neural Audio Compression
• Emerging Properties in Unified Multimodal Pretraining
• Stable Audio Open
• MT3: Multi-Task Multitrack Music Transcription
• ACE-Step 1.5: Pushing the Boundaries of Open-Source Music Generation
• Frequency-Aware Self-Supervised Music Representation Learning
• SongFormer: Scaling Music Structure Analysis with Heterogeneous Supervision
• Music Transformer
• The Platonic Representation Hypothesis
• DiffRhythm 2: Efficient and High Fidelity Song Generation via Block Flow Matching
• Adam: A Method for Stochastic Optimization

Related papers

• Rethinking Entropy Allocation in LLM-based ASR: Understanding the Dynamics between Speech Encoders and LLMs
• X-VC: Zero-shot Streaming Voice Conversion in Codec Space
• Autoregressive Guidance of Deep Spatially Selective Filters using Bayesian Tracking for Efficient Extraction of Moving Speakers
• VM-ARRAYDPS: Virtual Microphone Augmented Diffusion Posterior Sampling for Unsupervised Blind Speech Separation
• Conditional Flow Matching for Visually-Guided Acoustic Highlighting
• Anonymization, Not Elimination: Utility-Preserved Speech Anonymization