research area Data Science and Annotation, research area Speech and Natural Language Processing
content type paperpublished August 2026

Progressive Refinement: An Iterative Pseudo-Labeling Approach for Mandarin-English Code-Switching ASR

AuthorsQu Yang†**, Cakra Wardhana, Tim Ng

View publication

Code-switching (CS), alternating languages within the same utterance, poses significant challenges for automatic speech recognition (ASR) due to limited CS training data. This paper applies an iterative pseudo-labeling training approach to CS-ASR for the first time, demonstrating its effectiveness in leveraging unlabeled data to improve CS-ASR performance. The approach comprises three phases: pseudo-label generation, two-stage bilingual model training, and iterative improvements. It begins by generating pseudo-labels from a large unlabeled corpus, creating a semi-supervised dataset. This dataset supports a two-stage training framework where the model is pre-trained and then fine-tuned on supervised CS data. Iterative refinements further enhance the model’s accuracy in handling complex CS scenarios. Our approach significantly advances CS-ASR systems, achieving notable Mix Error Rate (MER) reductions on SEAME’s devman (6.35%) and devsge (8.29%) subsets.

  • † National University of Singapore
  • ** Work done while at Apple

Related readings and updates.

Code-Switching Detection via Language-Selective Dimensions in Speech Foundation Models

October 14, 2026research area Methods and Algorithms, research area Speech and Natural Language Processing

Code-switching (CS), the alternation between languages within an utterance, is common in multilingual speech but underrepresented in pretraining corpora. Increasing CS coverage requires finding relevant utterances in large speech collections so that they can be included in training data. One way to find such utterances is to transcribe the speech and detect language alternation in the text, but this depends on obtaining accurate multilingual…

Read more

Joint Speech Transcription and Translation: Pseudo-Labeling with Out-of-Distribution Data

May 12, 2023research area Speech and Natural Language Processingconference ACL

Self-training has been shown to be helpful in addressing data scarcity for many domains, including vision, speech, and language. Specifically, self-training, or pseudo-labeling, labels unsupervised data and adds that to the training pool. In this work, we investigate and use pseudo-labeling for a recently proposed novel setup: joint transcription and translation of speech, which suffers from an absence of sufficient parallel data resources. We…

Read more