Authors: Ali Syed¹; Kirsten Wagener¹; Volker Hohmann²; Michael Richter³
¹Hörzentrum Oldenburg gGmbH; ²Carl von Ossietzky Universität Oldenburg; ³Liverpool John Moores University
Background: Standard clinical assessments often lack the ecological validity necessary to measure cognitive load in complex soundscapes. To address this, a systematic framework was developed integrating Virtual Acoustics (VA), AI speech synthesis, and neuroimaging to objectively quantify listening effort in realistic environments.
Method: Three occupational environments were simulated using TASCAR with custom reflectivities and binaural rendering. Target stimuli consisted of 36 sentences synthesized via ElevenLabs. To simulate vocal exertion, a 1-D Convolutional CycleGAN processed Mel-spectrograms to inject Lombard features while maintaining high intelligibility (STOI >0.85). Cortical hemodynamics are captured via HD-fNIRS during low-demand (neutral speech) and high-demand (Lombard speech) conditions. Data processing involves Recursive Least Squares (RLS) filtering and Diffuse Optical Tomography (DOT) for 3D cortical reconstruction.
Results: Technical validation confirmed low latency and high ecological validity. Preliminary analysis indicates that CycleGAN-generated Lombard conditions elicit higher hemodynamic activation than baseline, with mean peak HbO values ranging from 0.108 to 0.130 µM. Early DOT reconstructions show localized cortical responses that correlate strongly with subjective effort and NASA-TLX scores.
Conclusion: The integration of VA, CycleGAN-modified speech, and DOT provides a robust computational pipeline for evaluating listening effort. This framework enables advanced assessments of hearing interventions within ecologically valid environments, bridging the gap between clinical testing and real-world auditory experiences.
