BRAU-Net++: U-Shaped Hybrid CNN-Transformer Network for Medical Image Segmentation

Accurate medical image segmentation is vital for clinical quantification, disease diagnosis, treatment planning, and other applications. Convolution-based U-shaped architectures excel at learning local features but rely heavily on image-specific inductive biases inherent to convolutions. Transformer-based models, on the other hand, effectively capture long-range dependencies using self-attention but face challenges of quadratic computational and memory demands as sequence lengths increase. To address these limitations, we propose BRAU-Net++, a hybrid CNN-Transformer network that integrates the strengths of both paradigms within a U-shaped architecture. The proposed BRAUNet++ adopts the two key ideas. First, it employs bi-level routing attention as its core building block to hierarchically construct the encoder-decoder structure, enabling efficient learning of global semantics while reducing computational complexity. Second, the network restructures skip connections by incorporating channel-spatial attention, which uses convolution operations to minimize spatial information loss during down-sampling and enhance multi-scale feature interactions. Extensive experiments on four diverse imaging modalities: Synapse, COVID-19, CT-ICH2020, and STS2D2023, demonstrate that BRAU-Net++ outperforms state-of-the-art methods, including its baseline BRAU-Net, under almost all evaluation metrics. These results reveal the model’s generality and robustness for multi-modal medical image segmentation tasks, while also highlighting that BRAU-Net++ achieves an optimal tradeoff between performance and efficiency by integrating dynamic sparse attention with a well-designed U-shaped architecture. Our code is publicly available on GitHub.