research area Methods and Algorithms, research area Speech and Natural Language Processing
content type paperpublished July 2026

Revisiting ASR Error Correction with Specialized Models

AuthorsZijin Gu, Tatiana Likhomanenko, Richard He Bai, Erik McDermott, Ronan Collobert, Navdeep Jaitly†**

View publication

Language models play a central role in automatic speech recognition (ASR), yet most methods rely on text-only models unaware of ASR error patterns. Recently, large language models (LLMs) have been applied to ASR correction, but introduce latency and hallucination concerns. We revisit ASR error correction with compact seq2seq models, trained on ASR errors from real and synthetic audio. To scale training, we construct synthetic corpora via cascaded TTS and ASR, finding that matching the diversity of realistic error distributions is key. We propose correction-first decoding, where the correction model generates candidates rescored using ASR acoustic scores. With 15x fewer parameters than LLMs, our model achieves 1.5/3.3% WER on LibriSpeech test-clean/other, outperforms LLMs, generalizes across ASR architectures (CTC, Seq2seq, Transducer) and diverse domains, and provides precise corrections in the low-error regime where LLMs struggle.

  • † Google
  • ** Work done while at Apple

Related readings and updates.

Delayed Fusion: Integrating Large Language Models into First-Pass Decoding in End-to-end Speech Recognition

January 18, 2025research area Speech and Natural Language Processingconference ICASSP

This paper presents an efficient decoding approach for end-to-end automatic speech recognition (E2E-ASR) with large language models (LLMs). Although shallow fusion is the most common approach to incorporate language models into E2E-ASR decoding, we face two practical problems with LLMs. (1) LLM inference is computationally costly. (2) There may be a vocabulary mismatch between the ASR model and the LLM. To resolve this mismatch, we need to…

Read more

Retrieval-Augmented Correction of Named Entity Speech Recognition Errors

September 11, 2024research area Knowledge Bases and Search, research area Speech and Natural Language Processing

In recent years, end-to-end automatic speech recognition (ASR) systems have proven themselves remarkably accurate and performant, but these systems still have a significant error rate for entity names which appear infrequently in their training data. In parallel to the rise of end-to-end ASR systems, large language models (LLMs) have proven to be a versatile tool for various natural language processing (NLP) tasks. In NLP tasks where a database…

Read more