Authors: Xavier Pérez Ruiz¹; Robert Hart¹; Waldo Nogueira¹
¹Hannover Medical School
Background: Cochlear implants (CIs) can restore sound perception, yet users struggle with speech understanding in noise. Existing speech enhancement methods often introduce latency and ignore CI coding characteristics. While end-to-end networks like DeepACE improve intelligibility without added latency, they focus on temporal envelopes. Little research has applied this approach to strategies utilizing temporal fine structure (FS), such as FS4, which mimics neural phase locking.
Method: We developed DeepFS4, an end-to-end version of FS4. We began with a DeepACE-like Temporal Convolutional Network (TCN) that performs envelope-based speech enhancement (SE). DeepFS4 was evaluated using objective measures and speech understanding tests in 10 CI recipients, including the German HSM sentence test, a MUSHRA quality rating, and a 3-AFC evaluation. To create the end-to-end version, an automatic gain control (AGC) and convolutional layers representing the FIR filterbank were integrated before the main TCN, with an additional parallel layer to emulate the zero-crossing detector for FS information.
Results: DeepFS4 achieved a mean signal-to-noise ratio improvement (SNRi) of 7 dB. In listening tests, it yielded significantly higher scores than both commercial FS4 and state-of-the-art front-end algorithms. Negligible differences in speech intelligibility were revealed when comparing different FS location variants (correctly placed vs. misplaced zero-crossings).
Conclusion: DeepFS4 demonstrates that integrating SE directly into sound coding via low-latency deep learning improves CI speech perception in noise. Robust improvements are possible even without FS location enhancements, validating an end-to-end architecture where SE is performed only on the envelope, bringing the field closer to AI-powered, clinically viable sound coding.
