High-Order Omni-Attention Paradigm for Remote Sensing Image Fusion

Multi-modal image fusion aims to comprehensively explore cross-modal information interactions between multiple source images, leveraging both spatial and channel dimensions. Existing methods primarily employ the cross-attention mechanism, which only exploits second-order properties in uni-dimensional modeling, thereby limiting higher-order interactions and omni-attention-enhanced learning capabilities. To address this, we propose a high-order omni-attention paradigm for multi-modal image fusion, adhering to the following principles: 1) High-order. Current promising methods stack cascaded self-attention-driven transformers to achieve high-order capabilities, resulting in substantial computational costs. Inspired by the self-attention saturation issue and its intrinsic first-order function, we argue against the necessity of computing attention scores in every cascaded layer. Instead, we perform attention operations in the initial layer and share the calculated attention scores across subsequent layers via attention transformations. This motivates us to shorten the resource-expensive cascaded attention chain within an effective high-order attention paradigm. 2) Omni-dimension. Regarding omni-dimensional attention score calculation, we reformulate each pixel in three-dimensional space as token and perform dense interactions through sequential element-wise spatial multiplication in the Fourier domain mathematically equivalent to global interactions, and rotation-based channel-wise self-attention mechanism simultaneously, to facilitate the discernment of inter-dependencies between source images. Inheriting the above merits, the tailored high-order core is capable of yielding omni-dimensional information interactions across spatial and channel axes simultaneously, culminating in three-dimensional modeling. Comprehensive experiments across various benchmarks in multi-modal image fusion tasks demonstrate that harnessing high-order interactions significantly enhances our model’s ability to exploit multi-modal synergies, leading to superior performance over state-of-the-art alternatives.