MuLaCover: Flexible Cover Song Generation via Symbolic Tonal Control

summary

The gist

A cover song re-renders an existing piece, most commonly by preserving its tonal content—melody and chord progression—while reshaping other musical attributes<ref:2610.26684#pg2> This suggests a…

In short

MULACOVER generates cover songs by conditioning a text-to-song model on a BPM-normalized symbolic lead sheet (melody and chords). This allows users to preserve the original song's tonal content while freely reshaping tempo, structure, and style through text prompts. It achieves state-of-the-art results competitive with commercial models like Suno.

Key concepts

Symbolic Lead Sheet
This is a representation of a song consisting of two parts: the melody line and the chord progression. Instead of using raw audio frames, this symbolic format captures the essential tonal content—what notes are played and what chords occur—making it easier to control the song's musical structure.
BPM Normalization
Before training, the lead sheet is time-stretched so that all sequences share a single, canonical tempo (e.g., 120 BPM). This crucial step removes any frame-level timing dependencies between the reference and the generated output, freeing the model to control its own pace regardless of how fast or slow it generates.
Alignment-Free Control
This technique enables the system to generate covers flexibly without needing precise frame-by-frame matching with a source recording. By focusing on tonal content rather than exact timing, users can achieve structural rearrangements and tempo variations while maintaining the song's core musical identity.

Terminology used across episodes

This episode discusses

The paper

MuLaCover: Flexible Cover Song Generation via Symbolic Tonal Control · Read on arXiv

MBZUAI University of Science and Technology Beijing University of the Arts MuLa Labs Vera Praxis Lab The Chinese University of Hong Kong Nanjing University of the Arts

A cover song re-renders an existing piece, most commonly by preserving its tonal content -- melody and chord progression -- while reshaping other musical attributes. This suggests a natural formulation for generative cover synthesis with pretrained music language models: explicitly control tonal content, while specifying lyrics and style through text. Existing approaches condition on frame-level features, tightly coupling generation to the reference's timing and structure, and limiting flexibility such as tempo variation, structural rearrangement, or partial conditioning. To this end, we present MuLaCover, a cover generation model that conditions a pretrained text-to-song foundation model on an editable symbolic lead sheet. Our design enables alignment-free control: generated outputs preserve the tonal signature of the source without frame-level alignment, allowing flexible timing, structure, and segment-level generation. We further introduce a training curriculum with partially mismatched audio-symbolic pairs, improving diversity and robustness. We evaluate MuLaCover with objective metrics, standard subjective ratings, and in-depth expert interviews that provide fine-grained diagnostic insights. Experiments show that MuLaCover achieves state-of-the-art performance among open-source systems and is competitive with leading commercial models such as Suno-v5.5. Audio samples and code are available at our project page: https://mulalabs.ai/mulacover/

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: We're looking at the paper: "MuLaCover: Flexible Cover Song Generation via Symbolic Tonal Control".

Jane: Here's the gist of the paper. MULACOVER presents a cover song generation model that conditions a pretrained text-to-song foundation model on a BPM-normalized symbolic lead sheet via a gated cross-attention module This design enables alignment-free control, allowing generated outputs to preserve the tonal signature of the source without frame-level alignment The system achieves state-of-the-art performance among open-source systems and is competitive with leading commercial models such as Suno-v5.5 Pruning redundant reasoning tokens A cover song re-renders an existing piece by preserving its tonal content—melody and chord progression—while reshaping other musical attributes This suggests a natural formulation for generative cover synthesis with pretrained music language models: explicitly control tonal content, while specifying lyrics and style through text Existing approaches condition on frame-level features, tightly coupling generation to the reference’s timing and structure, and limiting flexibility such as tempo variation, structural rearrangement, or partial conditioning The authors argue that the right tonal-content representation is the symbolic lead sheet long used in music information retrieval

13–15: : a melody line paired with a chord progression Crucially, before feeding the lead sheet into the model, we time-stretch it so that all sequences share a canonical tempo This BPM normalization is what frees the conditioning signal from any frame-wise tie to the source recording How the training signal is built The symbolic lead sheet representation, which comprises a melody line and a chord progression, captures the tonal content essential to the cover task Given a reference recording, we extract the vocal melody and chord progression with pretrained transcription models To prevent the lead sheet from inheriting the source’s specific tempo, and thereby to avoid imposing any frame-wise alignment between c and the generated audio, we estimate the source’s beats-per-minute (BPM) and time-stretch the lead sheet so that all sequences are normalized to a canonical BPM=120 After BPM normalization, the melody is encoded as a binary pianoroll M ∈ 0, 1 T ×256, whose 256 = 128 × 2 feature dimensions capture the onset and sustain status of each of the 128 MIDI pitches at every step The chord progression is encoded as a binary chromaroll H ∈ 0, 1 T ×12 over the twelve pitch classes The two representations are projected into a shared embedding space and concatenated along the feature dimension to form the conditioning sequence consumed downstream: c = M WM∥H WH ∈ R T ×D, (1) Where the gains come from The system is trained in three stages, where Stage 3 exposes the model to cross-version pairings of content and style For Stage 3, we collect, for each song, a group of K available renditions and sample two indices i, j ∈ 1..., K: the tonal-content signal c is extracted from rendition vi, while the remaining controls l, s and the audio target are taken from rendition vj This process turns a generic text-to-song model into a cover generator through cross-cover finetuning on a curated paired dataset The evaluation complements listening-study questionnaires with a structured expert-interview protocol that elicits diagnostic, comparative judgments from trained listeners Pruning redundant reasoning tokens The Gated Adaptive Cross-Attention (GACA) module injects the symbolic condition c only into the last L layers of the backbone — layers N−L+1 through N The internal structure of the resulting GACA block is illustrated in Figure 1(b) A learned gate gl ∈ (−1, 1)S×D lets the model decide adaptively, for each position and feature dimension, how much symbolic information to admit at layer l The default (α, β, ρ)=(0, 1, 0) recovers the full-song setting Where the gains come from The evaluation shows that MULACOVER achieves state-of-the-art performance among open-source systems and is competitive with the leading closed-source commercial system Suno-v5.5 Under paired block-level Wilcoxon tests, MULACOVER significantly outperforms open-sourced baselines on Style Controllability, Creativeness, and Overall Musicality while showing no significant difference from Suno-v5.5 Pruning redundant reasoning tokens MULACOVER is alignment-free, supporting flexible tempo and partial-reference generation in a single unified model The paper concludes that MULACOVER outperforms open-source cover baselines on most metrics and is on par with the closed-source commercial system Suno-v5.5 This demonstrates that adding fine-grained controls does not compromise generation quality Where the gains come from MULACOVER is robust to moderate lead-sheet extraction errors in terms of generation quality, with both AudioBox and SongEval scores exhibiting only limited degradation across all corruption levels Overall, MULACOVER is robust to moderate lead-sheet extraction errors in terms of generation quality The authors hypothesize that this improvement may be related to our training objective in Stage 3 Pruning redundant reasoning tokens MULACOVER is robust to moderate lead-sheet extraction errors in terms of generation quality, with both AudioBox and SongEval scores exhibiting only limited degradation across all corruption levels In contrast, cover identification performance decreases as the corruption level increases Interestingly, MuQ-Dist increases under corrupted conditions The authors hypothesize that noisy conditions introduce additional deviations from the original song, encouraging more diverse reinterpretations and thus increasing the distributional distance between the generated cover and the reference Where the gains come from MULACOVER is robust to moderate lead-sheet extraction errors in terms of generation quality, with both AudioBox and SongEval scores exhibiting only limited degradation across all corruption levels In contrast, cover identification performance decreases as the corruption level increases Interestingly, MuQ-Dist increases under corrupted conditions The authors hypothesize that noisy conditions introduce additional deviations from the original song, encouraging more diverse reinterpretations and thus increasing the distributional distance between the generated cover and the reference Where the gains come from

Title and authors: Tom: So we're looking at this paper from arXiv called "MuLaCover: Flexible Cover Song Generation via Symbolic Tonal Control". It’s about taking an existing song and re-rendering it by keeping the melody and the chords but letting you change everything else.

Jane: That sounds pretty simple, but I want to know how they're doing that—keeping the core musical structure while allowing for so much flexibility in style.

Lu: The paper proposes using a symbolic lead sheet as the main control signal for this, which is essentially just a melody line paired with a chord progression. This approach aims to separate the tonal content from the surface acoustics of the original recording.

Meng: So, if you use that symbolic lead sheet, what's actually happening in terms of how it conditions this text-to-song foundation model? What’s the mechanism there?

Tom: Well, they condition it using a gated cross-attention module. This design is key because it allows for alignment-free control. It means you don't need to line up every single frame between the source and what you're generating, which gives you much more freedom.

Jane: Alignment-free control sounds like it could really help with things like changing the tempo or rearranging parts of a song without getting stuck rigidly to the original timing.

Lu: Exactly. The system achieves this by first time-stretching the symbolic lead sheet so that all sequences share a canonical tempo, which they set at one hundred twenty beats per minute. This BPM normalization is what frees up that conditioning signal from being tied to any specific frame in the source recording, allowing it to be flexible.

Meng: So, how do they build this initial symbolic lead sheet? It sounds like a big step if you can extract just the melody and chords reliably from an existing audio file.

Tom: They use pretrained transcription models to extract both the vocal melody and the chord progression from a reference recording. This is the tonal content that defines what you want to preserve in the cover song.

Jane: And then they turn those extracted elements into two different binary representations: a pianoroll for the melody and a chromaroll for the chords, which are then projected into a shared embedding space before being concatenated.

Lu: That concatenation forms this conditioning sequence, which is what feeds into the model downstream. It’s essentially feeding it both musical elements as structured data, not just raw audio features.

Meng: What about the training process? I’ve seen so many models trained on single reference songs; how do they handle learning to switch between different versions of the same song for a cover task?

Tom: They use a three-stage training process. The third stage is where they expose the model to cross-version pairings of content and style. They take a group of K available renditions for each song and sample two indices, i and j.

Title and authors: Jane: So, in that third stage, they extract the tonal content signal from rendition i while taking the lyrics and style controls from rendition j. That’s how they teach it to mix things up creatively.

Lu: They’re essentially doing a cross-cover finetuning on this curated paired dataset. This process turns a generic text-to-song model into something that specifically understands the concept of covering, where you control the musical skeleton and style through text.

Meng: That sounds like a lot of data curation work to get those high quality pairs right. And what about the architecture itself? How do you actually make sure this symbolic information gets into the model at the right time?

Tom: They use a Gated Adaptive Cross-Attention module, or GACA. This module injects that symbolic condition only into the last L layers of a text-to-song backbone.

Jane: So they aren't flooding every layer with this musical information, which makes sense if you want to maintain general acoustic understanding in the lower layers while focusing the specific tonal control higher up.

Lu: The gate inside that module is what makes it adaptive; it lets the model decide how much symbolic information to admit at each position and feature dimension. By default, when things are set to zero, it recovers the full song setting.

Meng: That sounds like a neat way to manage complexity during inference, keeping the conditioning signal targeted. But what’s the result of this whole setup compared to what you’d get without that gating?

Tom: The evaluation shows MULACOVER achieves state-of-the-art performance among open-source systems and is competitive with the leading commercial system, Suno v5 point 5 <ref:2610.26684#pg1>. Under paired block-level Wilcoxon tests, it outperforms open-sourced baselines on style controllability and creativity, and shows no significant difference from Suno v5 point 5 in those areas <ref:2610.26684#pg1>.

Jane: That’s a strong comparison because it puts this open-source work directly against a very polished commercial system like Suno v5 point 5 <ref:2610.26684#pg1>. It suggests the control they have is genuinely effective at capturing musicality.

Lu: The results are interesting because they show that adding these fine-grained controls doesn't compromise generation quality at all; you get the cover, and it sounds good.

Meng: I’m interested in robustness here. How does this system handle things going wrong during the initial lead-sheet extraction? Because I know that’s a real problem when you try to automate music analysis.

Tom: The authors tested robustness to moderate lead-sheet extraction errors, and both AudioBox and SongEval scores showed only limited degradation across all corruption levels. That suggests the method is quite resilient to those kinds of mistakes.

Jane: That’s reassuring for real-world use; if the initial musical analysis isn't perfect, the cover generation doesn't immediately fall apart. But did they test when things get really messy?

Lu: They did test how identification performance changes as corruption increases, and they found that it actually decreases. Interestingly, the distributional distance between the generated cover and the reference increases under corrupted conditions.

Title and authors: Meng: So, noisy input makes it more likely to create a different version of the song rather than just producing a slightly broken one? That’s an interesting hypothesis for deployment.

Tom: Yeah, they hypothesize that noisy conditions introduce additional deviations from the original song, which encourages more diverse reinterpretations and thus increases the distributional distance between the generated cover and the reference.

Jane: So, when you look at this paper on MuLaCover: Flexible Cover Song Generation via Symbolic Tonal Control, what does this mean for us as listeners or creators who just want to remix music?

Tom: It means you can preserve the core musical DNA—the melody and chords—while using text prompts to completely reshape the style, tempo, instrumentation, or even the lyrics.

Lu: The main implication is shifting how we think about music generation: instead of just trying to generate a whole new song from scratch based on a prompt, you can treat tonal content as an explicit control signal that you can condition your model on directly.

Meng: From an engineering standpoint, it suggests that if you can successfully create high-quality symbolic representations like this lead sheet, the text-to-song foundation models become much more versatile tools for specific remixing tasks.

Jane: It sounds like a solid piece of work that tackles the challenge of control in generative music by focusing on what’s invariant—the tonality—and letting the rest be flexible.

Tom: Yeah, MULACOVER is showing that adding these fine-grained controls doesn't compromise generation quality at all when you have a robust training strategy like this cross-cover setup.

Lu: The combination of the BPM normalization and the GACA module seems to be the core mechanism that makes this alignment-free control work without needing frame-level precision.

Meng: So, for folks who are actually building these systems, it shows that you don't necessarily need perfect source audio alignment to get high-quality covers from existing music.

Jane: It’s a practical tool because it lets you take any song and put it through a creative filter defined entirely by text prompts.

Tom: We’re wrapping up our look at MuLaCover: Flexible Cover Song Generation via Symbolic Tonal Control. It shows that explicit control over tonal content, handled through a symbolic lead sheet, gives you the flexibility to reshape musical attributes without being locked into the source recording's exact timing.

Lu: The ability to accept partial references via attention masking on that normalized signal is what really opens up possibilities for more complex creative uses of these models.

Meng: It’s a solid paper because it connects a concrete representation—the lead sheet—to a powerful generation model in a way that seems practical for actual production pipelines.

Jane: We'll be keeping an eye on how this kind of explicit tonal control influences the next generation of music AI tools.

The paper's summary: Tom: So, MuLaCover is basically taking an existing song and letting you re-render it by keeping the melody and chords but letting you change all the other stuff using text prompts, like tempo or style.

Jane: That’s right, and what they're doing is focusing on that tonal content—the melody line and the chord progression—as the main control signal for a text to song foundation model.

Lu: They are using this symbolic lead sheet because it captures the essence of what you want to preserve, separating it from the specific way that original audio was recorded.

Meng: So, if we break down what they did in that training setup, they first extract the vocal melody and chord progression from a reference recording.

Jane: Right, and then crucially, before feeding that lead sheet into the model, they time-stretch it to a consistent tempo across all sequences.

Tom: That BPM normalization is key because it frees up that control signal from being tied to any specific frame in the source recording.

Lu: After normalizing the tempo, they encode both the melody and the chords into separate binary representations—a pianoroll and a chromaroll—which are then combined into one sequence.

Jane: That combined sequence is what conditions the text to song model downstream, giving it structured musical information instead of just raw audio features.

Meng: The training process is interesting because they use a three-stage approach where stage three involves cross version pairings of content and style.

Tom: They collect several renditions for each song, and in stage three, they sample two indices—one for the tonal content and one for the lyrics and style controls.

Lu: This cross cover finetuning teaches the model how to render one rendition’s tone with another rendition’s lyrics and style.

Jane: So what this means for us is that you can get a full song cover even when you only have a piece of the original music, as long as you give it that tonal content signal.

Meng: It also addresses robustness; they found that moderate errors in lead sheet extraction don't severely hurt generation quality, which is good for real-world use.

Tom: They also looked at what happens when the input data gets noisy, and they found that the distributional distance between the generated cover and the original reference actually increases under those noisy conditions.

Lu: The theory behind that is that noisy input introduces deviations from the original song, which encourages more diverse reinterpretations instead of just a slightly broken version.

Jane: This suggests that imperfection in our input could actually lead to more creative outcomes if the model is trained this way.

Tom: It’s a big deal because it shows that adding these specific controls doesn't compromise generation quality at all, even when you use open source systems like MULACover.

Lu: The Gated Adaptive Cross-Attention module they use only injects the tonal condition into the last few layers of the model, which keeps the lower layers focused on general acoustic representations.

Jane: And that gating mechanism lets the model decide how much symbolic information to admit layer by layer, which sounds like a very smart way to manage complexity during generation.

Meng: From a production standpoint, being able to accept partial references via attention masking on that normalized signal opens up possibilities for more complex remixing tasks.

Tom: So, the core idea is explicitly controlling the tonality while leaving the other musical attributes—tempo, structure—to be reshaped by text prompts.

Jane: It really shifts how we think about music generation from trying to create everything from scratch to conditioning a model on specific, explicit musical elements.

Lu: This paper connects a concrete representation, the lead sheet, directly to a powerful generation model in a way that seems practical for actual production pipelines.

Tom: So the next thing we need to look at is how this kind of explicit tonal control stacks up against those closed source systems like Suno v5 point five <ref:2610.26684#pg1>.

The paper's improvements: Tom: We're now looking at what the authors suggest are the specific improvements to their MULACover setup for better control over things like tempo and structure.

Jane: So, what are these suggested improvements? What’s actually new or better about this approach?

Lu: One big thing they point out is the flexibility it gives you: you can preserve the core tonal content, the melody and chord progression, while freely changing other musical attributes using text prompts.

Meng: That means you can change things like vocal timbre or instrumentation without having to re-engineer the entire audio from scratch.

Jane: It also tackles alignment issues directly with this idea of alignment-free control—meaning you don't need to line up every single frame perfectly with the source recording.

Tom: They suggest that if you normalize the symbolic lead sheet so all sequences share a common tempo, it frees up that control signal from being tied to any specific timing in the original audio.

Lu: That BPM normalization is what allows the model to set its own pace and accept any sub-segment as a partial reference through attention masking.

Jane: That's interesting because it makes generating covers much more adaptable, letting you use only parts of a song as a starting point for your remix.

Meng: They also discuss partial reference generation, which is super useful for when you only have a fragment of the source music but still want to generate a full cover.

Tom: And they mention this training curriculum, where they use partially mismatched audio and symbolic pairs to improve diversity and robustness during the training phase.

Lu: That's what we mean by exposing the model to those slightly imperfect pairings early on so it learns how to handle noise better later.

Jane: So these improvements focus a lot on making the system more flexible for real creative use rather than just getting a single, perfect output from one source song.

Meng: From an engineering standpoint, having this level of flexibility means the deployment pipeline is much simpler because you don't need to spend as much time aligning audio frames manually before generation.

Tom: And they highlight the Gated Adaptive Cross-Attention module again, showing that injecting that symbolic information only into the last few layers keeps the lower layers focused on general acoustic understanding.

Lu: That gating mechanism lets the model decide how much symbolic information to admit at each position and feature dimension, which is a smart way to manage complexity during generation.

Jane: This whole direction seems aimed at making these models better tools for remixers who want more granular control over the final product.

Meng: They also showed that they tested robustness against lead sheet extraction errors and found only limited degradation in generation quality, which is good for deployment because we know it won't break easily.

Tom: So, the takeaway here is that explicit control over tonal content via a symbolic lead sheet gives you the flexibility to reshape musical attributes without being locked into the source recording's exact timing.

Lu: This system seems really promising because it connects a concrete representation, like a lead sheet, directly to a generation model in this flexible way.

Jane: It’s going to be important for how we think about music AI moving toward more user-defined remixing tools.

Meng: So the next thing we need to look at is how this kind of explicit tonal control stacks up against those closed source systems like Suno v5 point five.

Conclusion: Tom: So we're wrapping up our discussion on MuLaCover: Flexible Cover Song Generation via Symbolic Tonal Control by summarizing what this whole paper means for music AI and giving our final thoughts on its implications.

Jane: Basically, they proved you can take an existing song and treat its melody and chords as a fixed tonal blueprint while letting text prompts dictate the style, tempo, and structure of the new version.

Lu: The main implication is that you don't need frame-level alignment to get good control over musical content; you just need that symbolic lead sheet representation.

Meng: For us in production, it means we can build pipelines where we can accept partial references—like just a snippet of a song—and still generate a coherent full cover.

Tom: They also showed that the Gated Adaptive Cross-Attention module is smart because it only injects that tonal condition into the later layers of the model, which keeps the lower layers focused on general sound representation.

Jane: And they confirmed that even if your initial song extraction has some errors, like a slightly off lead sheet, you still get decent generation quality.

Lu: The training strategy of using those partially mismatched audio and symbolic pairs really makes it robust to those kinds of input imperfections during the learning phase itself.

Meng: I just wonder about the cost for scaling this up; if we use this symbolic lead sheet method, does it make the inference process faster than our current frame-based models?

Tom: That’s a fair question, Meng. The speed depends on how efficiently that GACA module runs, but the main gain is in control quality and flexibility.

Jane: So we’re looking at a system that gives creators fine-grained control over musical identity while being forgiving of imperfect input data.

Lu: It connects a concrete representation, the lead sheet, to a generation model in a practical way that opens up more possibilities for user-defined remixing tasks.

Meng: It's certainly more robust than I expected, but we still need to see how much real traffic testing it handles before we can put this into our main production flow.

Tom: We definitely need to keep an eye on those real-world stress tests, because a lab result isn't the same as what happens when you have thousands of users hitting the system.

Jane: It’s exciting work because it shows that explicit control over tonality is a really viable path forward for more creative and flexible music AI tools.

Lu: I think this approach will push representation learning in music toward focusing on symbolic structures rather than just raw waveform analysis.

Meng: Yeah, the focus on the symbolic lead sheet is what makes it stand out compared to models that rely purely on frame-level audio input.

Tom: So that's our wrap-up for MuLaCover: Flexible Cover Song Generation via Symbolic Tonal Control, showing us a very powerful way to handle musical control.

Jane: Thanks for tuning in, and we’ll see what the next paper brings to the table next time.

More episodes

← Home