Reinforcement Learning for Data-Efficient Code-Switched ASR

summary

The gist

Audio-language models can be prompted for code-switched speech, but their decoding is not optimized for code-switching and often fails at language boundaries <ref:2607.02757#pg2> The paper proposes a…

In short

The research addresses how to adapt audio-language models for code-switched Automatic Speech Recognition (ASR) efficiently using reinforcement learning. They propose a method combining an error rate reward and a script fidelity reward, implemented via group relative policy optimization and a two-pass draft refinement procedure. This allows the model to learn transcription quality and correct writing systems with minimal training data.

Key concepts

CS-ASR
Code-switched ASR is the task of recognizing speech that mixes two or more languages within a single utterance. The challenge is that standard models struggle at language boundaries, leading to transcription errors in mixed speech.
GRPO
Group Relative Policy Optimization (GRPO) is a reinforcement learning technique used to train the model by learning from relative preferences among a small set of candidate transcriptions. It guides the model toward better transcription sequences based on learned rewards.
Script Fidelity Reward (SHR)
The Script Fidelity Reward penalizes incorrect writing systems in the output. It assigns a reward of 1 if every character in the predicted transcription belongs to an allowed script, effectively preventing 'hallucinations' where wrong characters or scripts are generated.

Terminology used across episodes

This episode discusses

The paper

Reinforcement Learning for Data-Efficient Code-Switched ASR · Read on arXiv

Ziwei Ye, Peter Vickers

Spotify Canada

DOI: 10.21437/Interspeech.2026-2667

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Reinforcement Learning for Data-Efficient Code-Switched ASR".

Jane: Here's the gist of the paper. Audio-language models can be prompted for code-switched speech, but their decoding is not optimized for code-switching and often fails at language boundaries <ref:2607.02757#pg2> The paper proposes a practical reinforcement learning with verifiable rewards recipe for dataefficient adaptation of audio-language models to code-switched ASR using group relative policy optimization <ref:2607.02757#pg4> This method combines an error rate reward with a script fidelity reward that penalizes wrong writing systems and a two-pass draft-and-refinement procedure <ref:2607.02757#pg2>.

Tom: Problem Framing The researchers frame CS-ASR as a verifiable reward problem, stating that given reference transcripts, one can compute automatic rewards and use RL to directly optimize sequence-level transcription quality <ref:2607.02757#pg2>.

Jane: They utilize an established, publicly available speech-LLM (Qwen2-Audio) as a controlled testbed to study how reward design and data efficiency interact for code-switched adaptation <ref:2607.02757#pg2>.

Tom: The goal is to improve both transcription accuracy and code-switch fidelity given an audio segment x, a language context C, and a reference transcript y∗ <ref:2607.02757#pg2>.

Paper summary: Tom: So we've talked about the core setup of this paper, and now I want to go over what they actually claim about their approach to making these models better. We need to focus on the practical application of the RLVR with GRPO framework.

Jane: Right, so we know they are using reinforcement learning with verifiable rewards that can be automatically checked, and they’re augmenting those transcription rewards with code-switch consistency signals. The main goal is to improve both transcription accuracy and code-switch fidelity simultaneously.

Lu: They set up the problem given an audio segment x, a language context C, and a reference transcript y star, where your goal is to improve both transcription accuracy and code-switch fidelity. They define the language context C as an instruction specifying the language pair, like "The audio contains speech mixing Japanese and English".

Meng: What about the actual mechanism they use for training? I need to know how they're sampling those candidate transcriptions. What is G?

Tom: They train with GRPO, which learns from relative preferences among a small set of sampled candidate transcriptions. For each input x and C, you sample G candidate transcriptions, denoted as hat y i from the current policy pi theta o.

Jane: So G is just the number of candidates they sample to learn from? And what do they do with those samples during the update process?

Lu: They compute rewards R of hat y i, y star, C equals negative character error rate between your guess and the reference plus beta times script fidelity.

Meng: So that means they are combining a standard error metric with a penalty for wrong writing systems into one objective function. How does that help the model learn?

Tom: The script fidelity reward is defined as one if all characters in your candidate transcript belong to the allowed scripts, and zero otherwise, and they use this to penalize wrong writing systems.

Jane: And then they use a two-pass draft and refinement procedure where pass one samples G drafts, computes rewards R of g, and updates the policy with GRPO loss L1. Then pass two constructs a prompt using the best draft and samples G refinements, updating the model with GRPO loss L2.

Lu: That sounds like they are training it to both transcribe and self-correct within a single training iteration, which is what they are aiming for in terms of efficiency. This method extends the RLVR idea by augmenting transcription rewards with code-switch consistency signals.

Meng: So, if I'm deploying this, does that mean the model needs an external critic to be running constantly to evaluate these complex rewards?

Tom: No, GRPO is a reinforcement learning method that normalizes those rewards within groups of sampled completions to avoid needing an explicit learned critic. It has been used in reasoning reinforcement learning, and it has been applied to monolingual ASR and audio question answering.

Jane: So the group relative policy optimization handles the reward normalization so you don't need that extra component?

Lu: Exactly, because it learns from relative preferences among those samples instead of needing an explicit learned critic. It's a technique that has been used in reasoning reinforcement learning, and it has been applied to monolingual ASR and audio question answering.

Meng: So what does this mean for deployment? If the model needs to be constantly evaluating these complex rewards, I need something that runs on real traffic at scale.

Tom: It means the training setup is designed to be practical, focusing on data efficiency and verifiable rewards rather than relying on a critic that's hard to keep up with in production environments.

Jane: So what we are talking about here is how they're structuring the training so it’s efficient enough for real-world use?

Conclusion: Tom: Okay, wrapping up this look at "Reinforcement Learning for Data-Efficient Code-Switched ASR," the authors are proposing this practical reinforcement learning with verifiable rewards recipe.

Jane: So what's the big picture they’re trying to communicate about how they can use these techniques for adapting audio language models?

Lu: The central point is that RLVR with GRPO enables highly data efficient adaptation of speech-LLMs to code-switched ASR. They show that this method can match LoRA supervised fine tuning trained on the full dataset across ten language pairs when you only use ten percent of the training data and about forty times fewer gradient steps.

Meng: So, what does that translate to for a production engineer? What's the practical impact?

Tom: It means you can get solid performance with much less data, and it shows that RLVR with GRPO has potential to mitigate both error types through that two-pass refinement paradigm.

Jane: The gains transfer zero shot from synthesized training data to human recorded code-switching speech.

Lu: This method proves that you can match the performance of LoRA supervised fine tuning trained on the full dataset across ten language pairs when you only use ten percent of the training data and about forty times fewer gradient steps.

Tom: It means you can get solid performance with much less data, and it shows that RLVR with GRPO has potential to mitigate both error types through that two-pass refinement paradigm.

More episodes

← Home