Using Automated Speech-to-Text AI to Evaluate Relative Benefit of Assistive Listening Systems in Real-World Environments

Authors: Justin R Burwinkel1*; Rachel Ellen Barrett2; Eric George1; Daniel Marquardt1; Kenneth Jensen1


2University of South Dakota


Background There are many factors that impact the effectiveness of hearing aids and assistive listening systems when they are used in ecological listening environments. We developed hearing aid firmware that allowed the devices to wirelessly stream input microphone and induction coil audio signals to a smartphone application for storage and subsequent analysis.

Method Binaural hearing aids were used to collect synchronous recordings from hearing aid microphone and induction coil inputs from various listening situations where induction hearing loops were available for public assistive hearing accessibility. The stored recordings were then analyzed offline using the output of the Google Speech-to-Text (STT) recognizer. The Google STT analysis was repeated on recordings obtained from a controlled laboratory setup and compared to speech intelligibility scores of hearing-impaired subjects (n=20).

Results The Google STT recognizer correctly identified 91.8% of the words in the speech-only track of the original QuickSIN. Hearing-impaired subjects (n=20) and the Google STT showed similar performances when detecting words from the telecoil recordings, which were obtained during sound-field presentation of the QuickSIN test in noise. Performance was notably poorer for both the hearing-impaired subjects and Google STT recognizer when using the microphone input modality.

Conclusion Public hearing assistive technology was shown to provide real-world SNR improvements equivalent to ~5-30dB. Boundary microphones were found to be less effective inputs to assistive hearing accessibility systems than podium and lapel microphones. Facility operators should install public hearing assistive technology (e.g., induction hearing loops) to reduce the effects of noise, distance, and reverberation to help facilitate equitable communication access. Audiologists should consider the apparent efficacy of hearing assistive technology and hearing aid compatibility when making recommendations.