HCMReg: A Pyramid Network with Hybrid Encoder and Cross Mamba Fusion for Unsupervised Medical Image Registration

Deformable image registration is fundamental to many medical image analysis tasks, and deep learning-based methods have been a major focus of research in medical image registration. Recently, selective structured State Space Models, like Mamba, have shown significant promise in modeling long-range dependencies with linear complexity, providing a potential solution to address the limitations of convolutional neural networks’ limited receptive fields and Transformer-based methods’ quadratic complexity. Single-stage registration methods usually generate deformation fields at a single resolution, which is insufficient to effectively capture the correspondences between image pairs and results in poor registration accuracy. Therefore, we propose a novel pyramid network with Hybrid encoders and Cross Mamba fusion for unsupervised deformable three-dimensional (3D) medical image Registration. Within the framework, we design a hybrid encoder that integrates the locality of convolution and the globality of Mamba to extract hierarchical features for the subsequent registration. In the decoding part, we propose the Cross Mamba Fusion Module to perform effective cross-feature fusion based on the single-input Mamba. The whole network progressively fuses the extracted features and generates deformation fields, warping the moving image towards the fixed image in a coarse-to-fine manner. The proposed method is validated on 4 public brain magnetic resonance imaging datasets (3 and 1 datasets are for internal and external validation, respectively). Experiment results demonstrate that our method outperforms eight state-of-the-art registration networks in terms of accuracy and generalization ability, illustrating the potential of our method for deformable registration of 3D medical images in medical image analysis.