Authors: Meike Span¹, Kiki van der Heijden¹
¹Radboud University
Background: Neural network models for audio often lack ecological validity by focusing on either acoustic processing or spatial processing, but rarely both. This study investigated the simultaneous encoding of speech and spatial features in GRAM, a binaural general-purpose audio representation model designed to perform a wide range of listening tasks, mimicking the human auditory pathway.
Method: GRAM was trained using self-supervision on binaural spectrograms of naturalistic sound scenes containing reverberation and background noise. We evaluated layer-wise performance on two keyword classification tasks and one sound localization task. To understand the underlying mechanisms, we applied EigenGradCAM (an explainability technique) to identify the specific spectral regions driving classification and localization decisions.
Results: Layer-wise analysis revealed that localization performance peaked early at intermediate layer 7, while keyword classification peaked at the final layer 12. Explainability analysis showed that classification performance correlated positively with a broad mid-frequency cluster (0.4–1.3 kHz). For localization, a focus on high-frequency regions (5.8–6.7 kHz) correlated positively with performance, whereas low-frequency (130–450 Hz) and mid-high regions (2.4–3.1 kHz) correlated negatively.
Conclusion: In a general-purpose audio model, spatial features are encoded earlier than speech features. This hierarchy mirrors expectations of auditory processing streams. Future research will compare GRAM representations with human neural activity to determine if similar computational trade-offs exist within the biological auditory brain.
