A Physiologically Explainable Machine Learning Framework for Hearing Aid Development

Background: Existing hearing aid devices provide only an approximate match to individual hearing needs, primarily amplifying sounds to restore basic audibility, but often falling short when it comes to enhancing speech understanding in difficult listening situations. In our previous work, we have proposed a machine learning framework that could overcome this shortcoming through integration of a physiological model of inner ear (Cascade of Asymmetric Resonators with Fast-Acting Compression model, CARFAC) and neural network. While initial results have been promising, the framework was only tested using pure tone stimuli. The neural network component of the previous framework was also a fully-connected layer, which makes robustness and explainability of the framework a challenging task when using more complex acoustic stimuli. In this work, we present the improved version of the framework while testing it against the speech stimuli.

Method: Figure 1 illustrates an overall view of the framework. Inputs to the framework are speech signals. During training, convolutional neural network (CNN) parameters have been tuned to minimize the difference between neural activity patterns (NAPs) generated by normal and impaired CARFAC models. Normal and impaired hearing have been emulated by Outer Hair Cell (OHC) health values of CARFAC. Comparison between normal (true) and impaired (predicted) NAPs is achieved through structural similarity index measure (SSIM), where loss value has been set to 1-SSIM.

Results and Conclusion: Convergence of the framework loss value has been observed using speech signals. After CNN parameters have been trained, impaired NAPs resemble normal NAPs, which suggests that the CNN could to some extent rectify the impairment caused by dysfunctional OHC. The CNN amplification role has also been explored through kernel visualisation, both in time and frequency domain. These results also indicate that this approach could provide for the development of hyper-personalised hearing aids by using models of an individual’s dysfunctional cochlea. The network architecture will be improved in future works to enhance framework performance in more challenging acoustic scenarios (e.g., speech in noise).