You Only Train Once: A Unified Framework for Both Full-Reference and No-Reference Image Quality Assessment

Existing Image Quality Assessment (IQA) models are limited to either full reference or no reference evaluation tasks, while humans can seamlessly switch between these assessment types. This motivates us to explore resolving these two tasks using a versatile model. In this work, we propose a novel framework that unifies full reference and no reference IQA. Our approach utilizes an encoder to extract multi-level features from images and introduces a Hierarchical Attention module to adaptively handle spatial distortions for both full reference and no reference inputs. Additionally, we develop a Semantic Distortion Aware module to analyze feature correlations between shallow and deep layers of the encoder, thereby accounting for the varying effects of different distortions on these layers. Our proposed framework achieves state-of-the-art performance for both full-reference and no-reference IQA tasks when trained separately. Furthermore, when the model is trained jointly on both types of tasks, it not only enhances performance in no-reference IQA but also maintains competitive results in full-reference IQA. This integrated approach facilitates a single training process that efficiently addresses both IQA tasks, representing a significant advancement in model versatility and performance.