research area Accessibility, research area Speech and Natural Language Processingconference Interspeech
content type paperpublished June 2025

Voice Quality Dimensions as Interpretable Primitives for Speaking Style for Atypical Speech and Affect

AuthorsJaya Narain, Vasudha Kowtha†, Colin Lea, Lauren Tooley, Dianna Yee, Vikramjit Mitra, Zifang Huang†, Miquel Espi Marques, Jon Huang, Carlos Avendano, Shirley Ren

View publication

Perceptual voice quality dimensions describe key characteristics of atypical speech and other speech modulations. Here we develop and evaluate voice quality models for seven voice and speech dimensions (intelligibility, imprecise consonants, harsh voice, naturalness, monoloudness, monopitch, and breathiness). Probes were trained on the public Speech Accessibility (SAP) project dataset with 11,184 samples from 434 speakers, using embeddings from frozen pre-trained models as features. We found that our probes had both strong performance and strong generalization across speech elicitation categories in the SAP dataset. We further validated zero-shot performance on additional datasets, encompassing unseen languages and tasks: Italian atypical speech, English atypical speech, and affective speech. The strong zero-shot performance and the interpretability of results across an array of evaluations suggests the utility of using voice quality dimensions in speaking style-related tasks.

  • † Work done while at Apple

Related readings and updates.

Affect Models Have Weak Generalizability to Atypical Speech

August 1, 2025research area Accessibility, research area Fairnessconference International Conference on Affective Computing and Intelligent Interaction (ACII)

Speech and voice conditions can alter the acoustic properties of speech, which could impact the performance of paralinguistic models for affect for people with atypical speech. We evaluate publicly available models for recognizing categorical and dimensional affect from speech on a dataset of atypical speech, comparing results to datasets of typical speech. We investigate three dimensions of speech atypicality: intelligibility, which is related...

Read more

Advancing Speech Accessibility with Personal Voice

December 18, 2023research area Speech and Natural Language Processing

A voice replicator is a powerful tool for people at risk of losing their ability to speak, including those with a recent diagnosis of amyotrophic lateral sclerosis (ALS) or other conditions that can progressively impact speaking ability. First introduced in May 2023 and made available on iOS 17 in September 2023, Personal Voice is a tool that creates a synthesized voice for such users to speak in FaceTime, phone calls, assistive communication apps, and in-person conversations.

Read more