Primary Menu

Education, Events, Publication

Funding & Recognition

Evaluation of Automated Transcription for Language Sample Analysis: The Impact of Accent on Word Error Rates

Year: 2023


Presenter Name: Haven Broadhurst

Description
Language sample analysis is considered a gold-standard procedure when assessing language disorders. It is an unbiased, reliable way to provide information regarding an individual's language abilities. Although Speech Language Pathologists would consider it a critical part of the assessment, the procedure can be very time-consuming. Recently, research evaluating the clinical application of automatic speech recognition software (ASR), Google Cloud Speech (GCS), for the purpose of transcription has been conducted on monolingual English-speaking children. The current study expanded upon these findings, by assessing the use of GCS with bilingual Spanish-speaking children, to examine the effect accented speech has on ASR transcription accuracy. To do this, audio samples elicited from school-aged bilingual children were transcribed with GCS and then evaluated for accuracy. 54 oral narrative samples elicited from Spanish-speaking bilingual children between the ages of 6;0-10;11 were transcribed by hand, and automatically, using Google Cloud Speech (GCS). A weighted word-error rate was used to calculate the minimum edit distance between the gold-standard transcripts and those transcribed with GCS, as a measure of transcription error. On average, the weighted word-error rate was 0.38 (SD = 0.15), meaning that 38% of words were incorrectly transcribed. The range of error rates was quite large. A follow-up analysis was conducted to determine whether there were significant differences in the transcription error of audio files elicited from children with perceived accents and those without. Results of the analysis indicated that there was not a statistically significant difference in transcription error for accented (M = 0.37, SD = 0.08) and non-accented (M = 0.39, SD = 0.19) audio samples controlling for age and gender of the narrator. The implications of these findings are that GCS may be a useful tool for SLPs to use in assessment of accented language.
University / Institution: Utah State University
Type: Poster
Format: In Person
Presentation #A43
SESSION A (9:00-10:30AM)
Area of Research: Education
Faculty Mentor: Sandra Gillam