Authors: Mark Saddler1, Torsten Dau1, Josh McDermott2
1Technical University of Denmark
2Massachusetts Institute of Technology
Background: Computational models that perform real-world tasks using simulated cochlear input could help link the peripheral effects of hearing loss to real-world perceptual consequences. Artificial neural networks optimized separately for sound localization and recognition tasks have been shown to account for many aspects of normal hearing human behavior. Here, we extend this approach to hearing loss using a model jointly optimized for multiple tasks.
Methods: We trained a single model to localize and recognize speech, voices, and environmental sounds from simulated auditory nerve representations of naturalistic scenes. Once trained, we compared the model’s speech recognition and spatial hearing to that of humans. We also measured psychoacoustic thresholds from the model by training linear classifiers to make binary judgments using the model’s learned features. To investigate the perceptual consequences of hearing loss, we altered the model’s peripheral input and measured effects of these alterations on behavior. Different types of hearing loss were simulated by manipulating the functionality and number of hair cells and auditory nerve fibers.
Results: When equipped with healthy cochleae, the model accounted for several aspects of binaural speech perception in humans with normal hearing, reproducing effects of noise, reverberation, and spatial separation between speech and noise. When healthy cochleae were replaced with damaged cochleae, the model’s performance characteristics resembled those of humans with hearing loss: speech recognition was degraded (especially at low SNRs) and spatial release from masking was reduced. Psychoacoustic thresholds measured from the model similarly reproduced patterns of normal and impaired human hearing.
Conclusions: Machine-learning-based models that generate behavior from simulated auditory nerve input predict some aspects of hearing-impaired behavior and have promise for linking peripheral damage to its perceptual consequences.
Hi Mark,
I watched your talk but didn’t get a chance to ask a question, well done! The model outcomes and analysis framework were all very interesting, have you published the results for hearing impairment anywhere, or planning to publish it soon? I found your latest preprint and I’ll read it soon, but didn’t see anything hearing-loss related there.
Thank you,
Fotis
Hi Fotis — thank you! So far we’ve written up the hearing loss modeling results as a Clarity Workshop paper: https://www.isca-archive.org/clarity_2025/saddler25_clarity.html .
Are there requirements relating to the data recorded, meaning…
1. Consent forms:human subjects
2. Maximum length of neural recording/max length of virtual auditory evoked potentials and hours, days, months, years
3. When documenting this data, neural recordings and the fake schemas or virtual noise, are your sources able to confirm this evidence?
4. If all the documented results and data were illegally obtained, through deceit or without a person’s knowledge, would it still be legit?
I feel that the digital age of the iot, biot, and other wireless communication systems will leave people vulnerable to breeches. Hacking the brain…sounds great to those who need brain help, but it’s not a stretch to think that in the wrong hands, especially without consent, anyone can create fake evoked potentials. Transmit the neural patterns of your daughter getting taped. The cops pounding on your door at 200am, an earthquake whatever. Maybe it’s not self talk, maybe those voices are virtual perceptions. If Yahoo, Chase, Universities and whole city public works departments are easily infiltrated or hacked…soon there shall be No free will.
Hi, thanks for your comment. I believe some of the concerns raised here might arise from a misunderstanding of our methods. We did not record or work with any human neural data in this project. Our model was only trained to recognize and localize sounds using simulated auditory nerve responses (as generated by a fairly classic signal processing description of how the ear encodes sound). We then evaluated how the trained model compares to human participants in various listening tests. The tests all involved playing listeners 2-second audio clips and asking them to perform a hearing task, such as recognizing words in different types of background noise. All participants provided informed consent and the MIT Committee on the Use of Humans as Experimental Subjects (COUHES) approved all experiments.