Viewport-Unaware Full-Reference Omnidirectional Image Quality Assessment With Inter-Patch and Sequence Similarity

Full-reference (FR) image quality assessment (IQA) (FR-IQA) has been extensively explored in the past two decades and is one of the most basic and hot topics in the image processing community, due to its indispensable role in quantitatively describing image quality degradation and guiding algorithm and system optimization. However, FR omnidirectional image quality assessment (OIQA) (FR-OIQA) has achieved less success, due to the natural gap between 2D images and omnidirectional images (OIs). To this end, we present a novel FR-OIQA model with Inter-Patch and Sequence Similarity (IPSS). Specifically, to avoid the extra computational load of viewport generation/prediction methods, IPSS processes OIs in aviewport-unawaremanner,i.e., directly extracting a patch sequence from an OI in the format of Equirectangular Projection (ERP) with retaining regions of interest. Furthermore, since the patches from ERP image contain inborn geometry deformation, thedeformation-awareconvolution is plugged into feature extraction and used to distill quality-aware features from theintrinsic pseudo-degradation, which are then utilized to measure inter-patch similarity. Finally, a distortion-aware interaction module is used to aggregate patch-wise quality-aware features, whose output is used to calculate patch-sequence similarity,i.e., the global quality of OI. Through comprehensive experiments on a large-scale OIQA database, we demonstrate the superiority of the proposed IPSS and the effectiveness of each module.