Authors: Rebecca Vos¹, Simone Graetzer¹, Gerardo Roa Dabike², Michael Akeroyd³, Scott Bannister⁴, Jon Barker², Trevor Cox¹, Bruno Fazenda¹, Jennifer Firth³, Alinka Greasley⁵, William Whitmer³
¹University of Salford; ²University of Sheffield; ³University of Nottingham; ⁴University of Manchester; ⁵University of Leeds
Background: Understanding lyrics is a key component of music enjoyment, yet hearing loss and hearing aid processing often significantly impair this ability. Predicting lyric intelligibility is complex due to variations in vocal style, genre, mixing, and listener hearing profiles. Existing speech intelligibility metrics often fail when applied to sung speech. The first Cadenza Lyric Intelligibility Prediction challenge (CLIP1) invited researchers to develop models that improve upon baseline predictions of lyric intelligibility for listeners with various levels of hearing loss.
Method: Participants were tasked with predicting the intelligibility of lyrics from musical excerpts (5–10 words). A dataset of 11,072 listener-labeled samples was provided. Two baseline systems were established: a non-intrusive Short-Term Objective Intelligibility (STOI) metric paired with a source separation model (HDemucs), and a Whisper-based Automatic Speech Recognition (ASR) system.
Results: Twenty-two teams submitted 27 systems, with 16 outperforming the Whisper-based baseline. Key technical trends included the use of foundation models for acoustic representation (24 systems), text encoders like Hugging Face T5 (4 systems), and additional features such as MFCCs, SNR, and pitch. Four systems utilized data augmentation to improve robustness. Despite these improvements, high error rates persist, particularly linked to variability in singing styles and musical accompaniment.
Conclusion: CLIP1 demonstrates that while foundation models and ASR-based approaches are promising, predicting lyric intelligibility remains a significant challenge. To address these limitations, the CLIP2 challenge is announced, featuring a larger dataset of thousands of AI-generated song excerpts to simulate a broader range of listening scenarios and genres.
CLIP2 Timeline: Launch: June 1, 2026 | Submission: November 1, 2026.
