Revisiting the Relation Between Language Model Perplexity and ASR Word Error Rate for Modern End-to-End Speech Recognition
The study re-examined how language model perplexity (PPL) relates to word error rate (WER) for modern end-to-end ASR systems, challenging old linear assumptions. Findings show this relationship varies by system type and context length. For attention-based models, internal language modeling subtraction is crucial for understanding external LM impact. LLM analysis reveals complexities in interpreting word-level PPL across different tokenization methods.