F&S-Net: A Dual Mission (Fusion and Super-Resolution) Framework Under Various Input Resolution

The intelligent fusion of heterogeneous source data for presentation or subsequent processing, is a popular research topic in computer vision society. While in most of current researches, the intrinsic resolution distinction among diverse input sources is commonly ignored. But in fact, identical-resolution is a relatively harsh prerequisite for various hetero data collecting hardwares and rarely happens in real-world applications. At the same time, as a typical knowledge-underdetermined research topic, super-resolution (SR) is deeply trapped by the shortage of available source information. In order to simultaneously cope with both mentioned dilemmas, in this paper, we proposed F&S-Net, a novel generic dual mission integration and co-prosperity framework. It is a unified architecture that compactly and organically integrates both distinct input resolution hetero fusion and heterogeneous assisting super-resolution mission together. Besides the evident efficiency benefits gained through two-in-one architecture merging, this integration is mutually boosting and could achieve performance win-win: First, embedding SR mechanism could effectively alleviate resolution distinction challenge faced by generic hetero image fusion; Second, the enrolling of paired input sources could directly double the amount of available information for the source-thirsty SR mission; Third, the shared components could be co-trained by both missions' data to achieve further robustness promotion. As to the structural innovation, facing doubled data and mission complexity, we devised throughout dimension extension system and two-stage based organic integration architecture. Stage 1 is in charge of mission shared primal information analysis. While Stage 2 focuses on mission-specialized further knowledge understanding and merging. This structural layout could efficiently realize general mutual knowledge co-enhancement and individual-level exclusive concentration to each specific mission. Extensive experiments have been conducted on publicly available MSRS, M3FD, and Set5 dataset to verify the effectiveness. Source code will be available soon.