Authors: Henry Zhong¹, Jorg Buchholz¹, Simon Carlile², Richard Lyon²
¹Macquarie University
²Google Research (Australia)
Scene recognition in audiologically relevant environments for hearing aids is challenging in part because of the limitations of existing datasets. These datasets are often not public, incomplete, or lacking relevant labels for hearing aids. Systematic comparison of machine learning models is also complicated by different training and testing datasets, and deploying these models on resource-constrained edge devices is also a challenge. Our solution is two-fold: we introduce AHEAD-DS, a sound dataset designed for scene recognition of audiologically relevant environments, and YAMNet+, a sound recognition neural network model. AHEAD-DS is derived from several open source datasets. It aims to provide a standardised, publicly available dataset with consistent labels relevant to hearing aids, facilitating model comparison. YAMNet+ is designed for deployment on edge devices such as smartphones and wireless earphones, and serves as a baseline model for sound-based scene recognition. YAMNet+ achieved a mean average precision of 0.76 on the testing set of AHEAD-DS over 14 categories. We found that applying transfer learning from the pretrained YAMNet model was essential. We demonstrated real-time sound-based scene recognition capabilities on edge devices by deploying YAMNet+ to an Android smartphone. Even with a phone with modest specifications (Pixel 3 – released 2018), the model processes each second of audio in approximately 30 milliseconds with only tens of milliseconds of latency. This talk will provide an overview of the architecture, together with the approach to training and the toolchain required to deploy the final network on the smartphone. We also look at the challenges of using the model on real-world data collected by students in their day-to-day life.
