Stereo Chorus of Whispers: Perceptually-Augmented Ear-Specific Intelligibility Prediction

This paper presents Stereo Chorus of Whispers, the winning system of the Cadenza Lyric Intelligibility Prediction (CLIP) Challenge. Our approach extends a Whisper-based intelligibility prediction framework to stereo listening conditions by performing ear-specific Whisper decomposition using a heterogeneous ensemble of pretrained Whisper models. This design allows diverse ASR-based perceptual cues to be captured independently for each ear. The resulting representations are further enriched with complementary perceptual metrics, including STOI, ESTOI, PESQ, and DNSMOS, together with a hearing-level indicator reflecting the simulated listening severity. These multi-cue features are fused using a lightweight transformer-based regression model to estimate the final lyric intelligibility score. Experimental results show that the proposed system achieves RMSEs of 25.89 and 26.44 on the official validation and evaluation sets, respectively, and ranks first in the CLIP Challenge, demonstrating strong robustness for lyric intelligibility prediction in complex musical environments. Our code is available at https://github.com/jinlongbin/Stereo-Chorus-of-Whispers.