GameHear: gamification and real-time motion tracking for conditioned play audiometry

Authors: Emil Hansen1, Erik Kjærbøl2, Stefania Serafin1, Ali Adjorlu 1, Lone Percy-Smith2

1Aalborg University

2Copenhagen Hearing and Balance Center

Background Conditioned Play Audiometry (CPA) is a modification of pure tone audiometry for 3-5-year-old children, turning the hearing test into a playful game. Currently, mostly analog methods are used, such as putting a toy in a box, when a tone is perceived.

Method In this work we propose a novel methodology to perform CPA by combining camera based tracking and gamification techniques. It has been shown that up to one in every third child goes home with little to no results when performing traditional CPA, as the child does not engage in the test. This paper explores gamifying methods for CPA to address low patient cooperation. The current setup at the Rigshospitalet in Copenhagen was transformed into a 3D play audiometer controlled by the child’s movements using an Azure Kinect.
Specifically, the setup consists of a camera that tracks the child’s movement, two screen where the game is displayed and a computer that drives both the games and the camera. Audiologists from an outside booth can control the game and monitor the behaviour of the child.

Results Two computer games were designed, where children where either embodying a Ninja who was required to hit some target objects when they heard them, or trying to to hit floating soap bubbles. The audiologists evaluated the two prototypes using the think aloud method and non-participatory observations. After iterative testing with audiologists, the setup is now ready for testing with children. At the conference we will show an evaluation of the designed prototype with children 3-5 years old.