Pseudo-Reference Driven Non-Intrusive Speech Quality Assessment via Multi-Task Learning

Non-intrusive speech quality assessment (NISQA) is a foundational yet challenging task within the domain of speech processing, particularly when confronted with limited information and the absence of comparative references. Although there are various feature extraction mechanisms, from natural scene statistics to deep neural networks, performance bottlenecks still exist. To address these challenges, this study proposes an auditory hallucination-guided quality regression network. First, an auditory hallucination reference is generated from the degraded speech to compensate for the lack of real reference. Then, the auditory hallucination reference information is paired with the degraded speech information and used as input to the regression model to learn the perceptual difference to realize accurate quality prediction. In addition, this study implements a multi-task learning strategy, predicting four critical quality dimensions: noisiness, coloration, discontinuity, and loudness. This approach enables service providers to identify and understand the specific causes of speech quality degradation rapidly. The effectiveness of the proposed method is verified by experiments. The experimental results indicate that the proposed method can enhance the performance of baseline models by generating pseudo-auditory speech and can also accurately predict the Mean Opinion Score and scores across the four quality dimensions. The proposed method can provide valuable guidance to online conference service providers in downstream tasks. Further, the experimental results demonstrate the proposed method's superior performance regarding multiple assessment metrics. Particularly, on datasets containing real-world scenarios, the proposed method achieved a 19.9% reduction in the Root Mean Square Error (RMSE) compared to the state-of-the-art speech quality assessment benchmark, demonstrating its effectiveness in enhancing the accuracy of speech quality assessment.