Authors: Francesco Ganis*1 ; Ali Adjorlu1 ; Stefania Serafin1 ; Lone Percy-Smith2
Affiliations:
1 Aalborg University
2 Righospitalet
Background
Auditory nerve (AN) deficiency entails additional challenges in music perception and appreciation for cochlear implanted (CI) subjects. Training is a key aspect in the rehabilitation process to improve the hearing experience. Audio-Verbal Therapy (AVT) sessions mainly focus on speech training, and only recently audiologists have been including music in their practice.
Method
This study investigates the effect of gamified musical training in a case
study on a 7 years old child bilaterally implanted and that presents AN deficiency. We designed a musical video game with the goal of training recognition of different musical instruments, using sound and vibrations as main feedbacks. The development of the software was conducted with special attention on participatory design, involving both clinicians and the final user. The game is based on a memory-game task where the child is asked to match couples of instruments on a set of cards. At every interaction with a card, the instrument’s sound is reproduced and coupled with vibrotactile stimuli. The vibrations are conveyed through a Sony PlayStation 5 DualSense controller. The initial training consists of 6 × 30 minutes sessions over two weeks in which the child completes levels with increasing difficulty, supervised by the parents at home.
Evaluation
The video game has been handed out to the participant, and the test is currently running. The video game automatically collects the duration of each level’s gameplay, and number of interactions per card/instrument. In addition, we will collect qualitative data through an interview with the child and their parents. The hypothesis is that there is going to be noticeable improvement in recognition of musical instruments’ timbre over the training period.
Conclusion
We presented a gamified musical training based on a memory game task with focus on timbre recognition. The first training program is currently running, and soon we will be able to present the results.”
