Adaptive mixture-of-experts network for medical image fusion

Abstract Medical image fusion is an important computer vision task in intelligent medical systems. It aims to integrate complementary information from different imaging modes into more information-rich representations. Although recent deep learning-based methods have demonstrated progress, they still face many challenges in handling modal inconsistency and the flexibility of adaptive fusion of multimodal features. To address these issues, in this paper, we propose an adaptive mixture-of-experts network (AMoENet) for medical image fusion. AMoENet consists of a multi-perspective feature extraction (MFE) module and an Adaptive Expert (AdEx) module. Specifically, the MFE module is built to extract the feature information in the source image. It combines Transformer and attention to effectively handle remote dependencies while enhancing the expressive ability of local and frequency-domain features. Additionally, the AdEx module is built to dynamically allocate fusion weights to each expert. This method enables the weight distribution and feature processing to adapt to each other through multi-expert collaborative optimization, which obtains information-rich fused images. Experimental results on medical datasets have demonstrated that the proposed model achieves superior performance in quantitative metrics and visual quality, along with strong generalization capabilities. The source code and implementation details are available at https://github.com/zenghui11/AMoENet .