DBCFuse: Dual-Branch Cross-Scale Feature Decomposition for MRI-PET Fusion

Determining the histopathological boundaries of gliomas is critical for providing diagnoses and prognoses. A single magnetic resonance (MR) image cannot fully reflect the extent of a tumor. Fused MR and positron emission tomography (PET) images provide comprehensive metabolic functional status and structural information for tumor tissues, aiding in the detection of gliomas and the delineation of tumor boundaries. However, extracting functional metabolic and structural information from medical images remains a significant challenge. The emerging multimodal medical image fusion technique effectively integrates different imaging modalities for clinical diagnosis and surgical navigation purposes. Recent advancements in vision transformers and convolutional neural networks (CNNs) have markedly improved the performance of image fusion methods. However, the redundancy and specificity of information across different image modalities are often neglected, leading to inefficient multimodal feature extraction and fusion processes. To address this issue, this paper proposes a dual-branch cross-scale fusion network to achieve accurate medical image fusion. First, we extract global semantic features from low-frequency data and local semantic features from high-frequency data in PET and MR images. This is accomplished by using the long-range attention mechanism of a Lite Transformer (LT) block and a wavelet transform (WT)-based convolution mechanism. We introduce a correlation loss to strengthen the relationship between the global features and the diversity of the local detailed features across different modalities, thereby reducing the redundancy exhibited by the global features and improving the degree of local specificity in the fused images. Next, we utilize the LT block for cross-modal global feature fusion, wavelet convolution for local feature fusion, and a cross-scale feature fusion module (CSFFM) for integrating features at different scales, which helps prevent the loss of cross-modal fused image details and maintains consistency among the cross-scale features. Finally, experiments conducted on datasets acquired from Harvard demonstrate that our proposed method outperforms other approaches, achieving superior performance across seven metrics, with a particularly significant improvement of 0.71 in terms of the standard deviation (SD) metric.