Reinforcement Learning for Data-Efficient Code-Switched ASR
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "Reinforcement Learning for Data-Efficient Code-Switched ASR".
Jane: Here's the gist of the paper. Audio-language models can be prompted for code-switched speech, but their decoding is not optimized for code-switching and often fails at language boundaries <ref:2607.02757#pg2> The paper proposes a practical reinforcement learning with verifiable rewards recipe for dataefficient adaptation of audio-language models to code-switched ASR using group relative policy optimization <ref:2607.02757#pg4> This method combines an error rate reward with a script fidelity reward that penalizes wrong writing systems and a two-pass draft-and-refinement procedure <ref:2607.02757#pg2>.
Tom: Problem Framing The researchers frame CS-ASR as a verifiable reward problem, stating that given reference transcripts, one can compute automatic rewards and use RL to directly optimize sequence-level transcription quality <ref:2607.02757#pg2>.
Jane: They utilize an established, publicly available speech-LLM (Qwen2-Audio) as a controlled testbed to study how reward design and data efficiency interact for code-switched adaptation <ref:2607.02757#pg2>.
Tom: The goal is to improve both transcription accuracy and code-switch fidelity given an audio segment x, a language context C, and a reference transcript y∗ <ref:2607.02757#pg2>.
Paper summary: Tom: So we've talked about the core setup of this paper, and now I want to go over what they actually claim about their approach to making these models better. We need to focus on the practical application of the RLVR with GRPO framework.
Jane: Right, so we know they are using reinforcement learning with verifiable rewards that can be automatically checked, and they’re augmenting those transcription rewards with code-switch consistency signals. The main goal is to improve both transcription accuracy and code-switch fidelity simultaneously.
Lu: They set up the problem given an audio segment x, a language context C, and a reference transcript y star, where your goal is to improve both transcription accuracy and code-switch fidelity. They define the language context C as an instruction specifying the language pair, like "The audio contains speech mixing Japanese and English".
Meng: What about the actual mechanism they use for training? I need to know how they're sampling those candidate transcriptions. What is G?
Tom: They train with GRPO, which learns from relative preferences among a small set of sampled candidate transcriptions. For each input x and C, you sample G candidate transcriptions, denoted as hat y i from the current policy pi theta o.
Jane: So G is just the number of candidates they sample to learn from? And what do they do with those samples during the update process?
Lu: They compute rewards R of hat y i, y star, C equals negative character error rate between your guess and the reference plus beta times script fidelity.
Meng: So that means they are combining a standard error metric with a penalty for wrong writing systems into one objective function. How does that help the model learn?
Tom: The script fidelity reward is defined as one if all characters in your candidate transcript belong to the allowed scripts, and zero otherwise, and they use this to penalize wrong writing systems.
Jane: And then they use a two-pass draft and refinement procedure where pass one samples G drafts, computes rewards R of g, and updates the policy with GRPO loss L1. Then pass two constructs a prompt using the best draft and samples G refinements, updating the model with GRPO loss L2.
Lu: That sounds like they are training it to both transcribe and self-correct within a single training iteration, which is what they are aiming for in terms of efficiency. This method extends the RLVR idea by augmenting transcription rewards with code-switch consistency signals.
Meng: So, if I'm deploying this, does that mean the model needs an external critic to be running constantly to evaluate these complex rewards?
Tom: No, GRPO is a reinforcement learning method that normalizes those rewards within groups of sampled completions to avoid needing an explicit learned critic. It has been used in reasoning reinforcement learning, and it has been applied to monolingual ASR and audio question answering.
Jane: So the group relative policy optimization handles the reward normalization so you don't need that extra component?
Lu: Exactly, because it learns from relative preferences among those samples instead of needing an explicit learned critic. It's a technique that has been used in reasoning reinforcement learning, and it has been applied to monolingual ASR and audio question answering.
Meng: So what does this mean for deployment? If the model needs to be constantly evaluating these complex rewards, I need something that runs on real traffic at scale.
Tom: It means the training setup is designed to be practical, focusing on data efficiency and verifiable rewards rather than relying on a critic that's hard to keep up with in production environments.
Jane: So what we are talking about here is how they're structuring the training so it’s efficient enough for real-world use?
Conclusion: Tom: Okay, wrapping up this look at "Reinforcement Learning for Data-Efficient Code-Switched ASR," the authors are proposing this practical reinforcement learning with verifiable rewards recipe.
Jane: So what's the big picture they’re trying to communicate about how they can use these techniques for adapting audio language models?
Lu: The central point is that RLVR with GRPO enables highly data efficient adaptation of speech-LLMs to code-switched ASR. They show that this method can match LoRA supervised fine tuning trained on the full dataset across ten language pairs when you only use ten percent of the training data and about forty times fewer gradient steps.
Meng: So, what does that translate to for a production engineer? What's the practical impact?
Tom: It means you can get solid performance with much less data, and it shows that RLVR with GRPO has potential to mitigate both error types through that two-pass refinement paradigm.
Jane: The gains transfer zero shot from synthesized training data to human recorded code-switching speech.
Lu: This method proves that you can match the performance of LoRA supervised fine tuning trained on the full dataset across ten language pairs when you only use ten percent of the training data and about forty times fewer gradient steps.
Tom: It means you can get solid performance with much less data, and it shows that RLVR with GRPO has potential to mitigate both error types through that two-pass refinement paradigm.
Ziwei Ye, Peter Vickers
Spotify Canada
cs.CL, cs.SD
Submitted: 2026-07-02
Updated: 2026-07-02
Comments: Accepted at Interspeech 2026
Journal ref: Proceedings of Interspeech 2026, pp. 6514-6518
DOI: 10.21437/Interspeech.2026-2667
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 90/100
The gist: Audio-language models can be prompted for code-switched speech, but their decoding is not optimized for code-switching and often fails at language boundaries <ref:2607.02757#pg2> The paper proposes a…
Key concepts
- CS-ASR
- Code-switched ASR is the task of recognizing speech that mixes two or more languages within a single utterance. The challenge is that standard models struggle at language boundaries, leading to transcription errors in mixed speech.
- GRPO
- Group Relative Policy Optimization (GRPO) is a reinforcement learning technique used to train the model by learning from relative preferences among a small set of candidate transcriptions. It guides the model toward better transcription sequences based on learned rewards.
- Script Fidelity Reward (SHR)
- The Script Fidelity Reward penalizes incorrect writing systems in the output. It assigns a reward of 1 if every character in the predicted transcription belongs to an allowed script, effectively preventing 'hallucinations' where wrong characters or scripts are generated.
Terminology
Summary
Audio-language models can be prompted for code-switched speech, but their decoding is not optimized for code-switching and often fails at language boundaries <ref:2607. p. 002757#pg2> The paper proposes a practical reinforcement learning with verifiable rewards recipe for dataefficient adaptation of audio-language models to code-switched ASR using group relative policy optimization <ref:2607. p. 002757#pg4> This method combines an error rate reward with a script fidelity reward that penalizes wrong writing systems and a two-pass draft-and-refinement procedure <ref:2607. p. 002757#pg2>.
Problem Framing
The researchers frame CS-ASR as a verifiable reward problem, stating that given reference transcripts, one can compute automatic rewards and use RL to directly optimize sequence-level transcription quality <ref:2607.02757#pg2>. They utilize an established, publicly available speech-LLM (Qwen2-Audio) as a controlled testbed to study how reward design and data efficiency interact for code-switched adaptation <ref:2607. p. 002757#pg2>. The goal is to improve both transcription accuracy and code-switch fidelity given an audio segment x, a language context C, and a reference transcript y∗ <ref:2607. p. 102757#pg2>.
How the training signal is built
The model is trained with GRPO, a RL method that learns from relative preferences among a small set of sampled candidate transcriptions <ref:2607.02757#pg4>. Rewards are computed as R(ˆyi, y∗, C) = −CER(ˆyi, y∗) + βsf ·Script(ˆyi, C), where the script fidelity reward penalizes wrong writing systems <ref:2607. p. 002757#pg2>. The script fidelity is defined as Script(ˆyi, C) = 1 if all characters in yˆi belong to the allowed scripts and 0 otherwise <ref:2607. p. 102757#pg4>.
Where the gains come from
The training procedure involves a two-pass draft and refinement procedure <ref:2607.02757#pg2>. Pass 1 samples G drafts, computes rewards R(g), and updates the policy with GRPO loss L1 <ref:2607. p. 102757#pg4>. Pass 2 constructs a prompt using the best draft and samples G refinements, updating the model with GRPO loss L2 <ref:2607. p. 102757#pg4>. This procedure trains the model to both transcribe and self-correct within a single training iteration <ref:2607. p. 102757#pg2>.
Performance Comparison
The results show that RLVR with two-pass refinement and the SHR reward achieves a micro-averaged CER of 0.147 on CS-FLEURS read test with only 20% of the training data, outperforming 100% LoRA SFT (0. p. 3159) <ref:2607. p. 302757#pg4>. The SHR reward substantially reduces script hallucinations (SHR 0. p. 3025 vs. 0.068 for CER-only at 10%) without degrading CER <ref:2607. p. 302757#pg4>. These gains transfer zero-shot from TTS-synthesized training data to human-recorded code-switching speech <ref:2607. p. 302757#pg2>. The largest RLVR gains appear on typologically distant pairs where code-switch transitions are hardest <ref:2607. p. 302757#pg4>.
Model Limitations and Strengths
RLVR wins on typologically distant pairs like ara-eng (0.393 vs. 0.652 for LoRA) and rus-eng, where LoRA's translation failure mode is most pronounced <ref:2607.02757#pg4>. Conversely, LoRA achieves more near-perfect transcriptions (29. p. 33% with CER. p. 0 RLVR produces fewer catastrophic errors (10. p. 31% with CER>0. p. 05 vs 11.8%), consistent with RL optimizing expected reward rather than memorizing individual examples <ref:2607. p. 302757#pg4>. The SHR reward is needed to constrain overcorrection on pairs where script boundaries are most ambiguous <ref:2607. p. 302757#pg4>. The two-pass refinement helps most pairs but consistently hurts ArabicEnglish on both datasets <ref:2607. p. 302757#pg2>.
Conclusion
The paper shows that RLVR with GRPO enables highly data-efficient adaptation of speech-LLMs to code-switched ASR <ref:2607.02757#pg2>. With only 10% of the training data and ∼40× fewer gradient steps, RLVR matches LoRA SFT trained on the full dataset across 10 language pairs <ref:2607. p. 302757#pg4>. A CER reward eliminates translation errors, while a dedicated SHR reward is needed to reduce script contamination, and a two-pass draft refinement paradigm has potential to mitigate both error types <ref:2607. p. 302757#pg2>. These gains transfer zero-shot from TTS-synthesized training data to human-recorded codewitching speech <ref:2607. p. 302757#pg2>. The paper demonstrates robustness across acoustic domains <ref:2607. p. 302757#pg4>.
Generative AI Use Disclosure
LLMs were used to produce a fraction of the source codes for experiment orchestration, and also for spellchecking and suggesting edits to further improve the fluency and conciseness of the paper during the drafting process <ref:2607.02757#pg2>. The authors state that LLMs were used for these purposes <ref:2607. p. 402757#pg2>.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Verification of Code-Switching Accuracy via Multi-Objective Reward Function: The improved system will use a reward function that combines an error rate reward with a script fidelity reward to
penalize wrong writing systems and a two-pass draft-and-refinement procedure.
This allows the model to simultaneously optimize for transcription accuracy (CER) and correct language usage/script adherence. -
Data Efficient Adaptation for Code-Switched ASR: The system can adapt audio-language models using only a small fraction of code-switched data, specifically
10% of the data matches LoRA supervised fine-tuning trained on the full dataset.
This significantly reduces the required labeled training set size while maintaining high performance. -
Robustness to Typologically Distant Code-Switching: The improved model will show
the largest gains on typologically distant pairs,
such as ara-eng and jpn-eng, where code-switch transitions are hardest, demonstrating superior generalization compared to methods like LoRA SFT. -
Zero-Shot Transfer to Unseen Code-Switching Corpora: The system can demonstrate
zeroshot transfer to SwitchLingua [12], a human-recorded codeswitching corpus,
meaning it can perform well on new, real-world code-switching data without further fine-tuning on that specific corpus. -
Enhanced Self-Correction Mechanism: The two-pass draft and refinement procedure allows the model to
train to both transcribe and self correct within a single training iteration,
mitigating early commitment errors where the model mightcommit early to an incorrect language mode.
Sources
- A Survey of Code-switched Speech and Language Processing
- Group Relative Policy Optimization for Speech Recognition
- Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM?
- Qwen2 Technical Report
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data