Music Pre-Processing Strategies to Enhance Cochlear Implant Users’ Enjoyment of Recorded Music

Authors: Lloyd May1, Blair Kaneshiro1, Aaron Hodges1, So Yeon Park1, Jonathan Berger 1

1Stanford University

Background: Cochlear implants (CIs) have significantly improved access to sound, especially speech, for a large user base, yet experiences with music remain unsatisfactory for many. Recent developments in AI pre-processing of music have shown promise by changing the relative loudness of various musical elements. This prompts an inquiry into whether the increased involvement of CI users in the music-mixing process could yield mutual benefits for both users and audio engineers (AEs).

Method: The study employed a mixed-methods approach by asking 10 CI users and 10 professional audio engineers (AEs) to complete multiple surveys, interviews, and mix multi-track recordings of diverse genres under various conditions, including with iterative feedback from a CI user. CI users then evaluated the various mixes using a CI-MUSHRA. Mixing strategies conducive to enhancing music enjoyment for CI users were identified, as well as various systemic barriers within the music industry, such as a lack of aural diversity training in most audio engineering programs.

Results: Analysis of collected data revealed two predominant mixing strategies favored by CI users: essentializing (removal of unwanted elements) and exaggeration (enhanced perception of desired musical elements). Participant responses also shed light on systemic ableism and audism within the music industry, with AEs expressing concerns about inadequate training in aural diversity considerations and experiencing anxiety regarding their own aural diversity. Overall, this study contributes valuable insights into CI user music listening preferences, approaches of AEs towards mixing for a diverse audience, and the impact of CI simulation, user feedback, and AE peer feedback on mix enjoyment by CI users. Drawing from these findings, the paper proposes design insights emphasizing the importance of customizable music listening tools that prioritize user agency and enjoyment over one-size-fits solutions.