CoSE-E: A Benchmark for Code-switched Speech Evaluation in Enterprise Settings
cs.SD, cs.CL, cs.LG, eess.AS
Submitted: 2026-09-28
Updated: 2026-09-28
Comments: Accepted to SALMA Workshop (Oral) at EMNLP 2026
Code: https://github.com/pipecat-ai/stt-benchmark
License: http://creativecommons.org/licenses/by/4.0/
The gist: Code-switching (CS), a seamless alternation between languages within a single utterance, remains a critical challenge in automatic speech recognition (ASR).
Terminology
Abstract
Code-switching (CS), a seamless alternation between languages within a single utterance, remains a critical challenge in automatic speech recognition (ASR). While prior works focus on conversational CS-ASR, enterprise settings demand evaluation of operational impact beyond edit-distance errors: how code-switching transcription errors propagate to downstream voice agent task failures. In this work, we propose (1) a CS-ASR synthetic benchmark and multidimensional evaluation framework tailored to enterprise domains, (2) systematic evaluation of frontier ASR systems across 5 language pairs, (3) diagnostic analysis of the additional transcription errors that code-switching introduces across language pairs and models. We release COSE-E to support enterprise-focused CSASR evaluation for multilingual voice agents in enterprise deployment.
Sources
- Code-Switching in End-to-End Automatic Speech Recognition: A Systematic Literature Review
- EVA-Bench: A New End-to-end Framework for Evaluating Voice Agents
- Voxtral
- Canary-1B-v2 & Parakeet-TDT-0.6B-v3: Efficient and High-Performance Models for Multilingual ASR and AST
- Gemma 4 Technical Report
- Qwen3-Omni Technical Report
Related papers
- Few-Shot Open-Set Audio Classification via Transductive Prototype Refinement and Class Logit Enhancement
- Spectral Masking and Interpolation Attack (SMIA): A Black-box Adversarial Attack against Voice Authentication and Anti-Spoofing Systems
- Remix the Timbre: Diffusion-Based Style Transfer Across Polyphonic Stems
- Harmonic Representations in Symbolic Music Transformers are Localized and Controllable
- AVMeme Exam: A Multimodal Multilingual Multicultural Benchmark for LLMs' Contextual and Cultural Knowledge and Thinking
- SoundWeaver: Compositional Warm-Starting for Text-to-Audio Diffusion Serving