Advancing LLM-based phoneme-to-grapheme for multilingual speech recognition

arXiv:2603.29217 · eess.AS, cs.CL, cs.SD · Submitted 2026-03-31 · Read on arXiv

eess.AS, cs.CL, cs.SD

Submitted: 2026-03-31

Updated: 2026-10-06

Comments: NCMMSC 2026 accepted version

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Robustness — 16× in this paper · explained in 5 episodes
• Fine-Tuning — 9× in this paper · explained in 5 episodes
• Ablation — 6× in this paper · explained in 1 episode
• Low-Resource Languages — 6× in this paper · explained in 4 episodes
• Accuracy — 5× in this paper · explained in 1 episode
• Sampling-K Marginalization — 4× in this paper
• Adaptation — 3× in this paper · explained in 1 episode
• Benchmark — 3× in this paper · explained in 2 episodes
• Large Language Models (LLMs) — 3× in this paper · explained in 6 episodes
• Adam — 2× in this paper · explained in 2 episodes
• Data Augmentation — 2× in this paper · explained in 7 episodes
• Data Augmentation with Noisy Phonemes — 2× in this paper
• Generative AI — 2× in this paper · explained in 2 episodes
• International Conference on Learning Representations — 2× in this paper
• Transformer — 2× in this paper · explained in 2 episodes
• Ablation study — 1× in this paper · explained in 1 episode
• Automatic Speech Recognition (ASR) — 1× in this paper · explained in 2 episodes
• Core — 1× in this paper · explained in 2 episodes
• DANP and Top-K Marginalization — 1× in this paper
• Embeddings — 1× in this paper · explained in 2 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Modularity — 1× in this paper · explained in 9 episodes
• Proposed Simplified Sampling-K Marginalization — 1× in this paper
• Tokenization — 1× in this paper · explained in 2 episodes
• Transfer Learning — 1× in this paper · explained in 10 episodes
• Two-Stage Architecture — 1× in this paper · explained in 2 episodes

Sources

• PolyIPA -- Multilingual Phoneme-to-Grapheme Conversion Model
• Qwen3 Technical Report

Related papers

• Rethinking Entropy Allocation in LLM-based ASR: Understanding the Dynamics between Speech Encoders and LLMs
• X-VC: Zero-shot Streaming Voice Conversion in Codec Space
• Autoregressive Guidance of Deep Spatially Selective Filters using Bayesian Tracking for Efficient Extraction of Moving Speakers
• VM-ARRAYDPS: Virtual Microphone Augmented Diffusion Posterior Sampling for Unsupervised Blind Speech Separation
• Conditional Flow Matching for Visually-Guided Acoustic Highlighting
• Anonymization, Not Elimination: Utility-Preserved Speech Anonymization