EW2A: Evolutionary Window Attention Adaptation for Medical Image Segmentation

Medical image segmentation is a critical yet challenging task due to the intricate anatomical structures, low contrast, and high noise levels inherent in medical images. Leveraging the inductive bias for spatial locality, window attention-based Transformer models have demonstrated significant promise in advancing image analysis tasks. However, the reliance on manually tuned window attention mechanisms limits their adaptability to varying feature modeling requirements and the unique challenges posed by medical imaging, resulting in suboptimal performance. In this paper, we propose evolutionary window attention adaptation (EW2A), a novel method specifically designed to overcome these challenges. In particular, a window attention-based supernet architecture is constructed firstly, where its search space focuses on two key aspects: optimal shift sizes for the window attention mechanism and effective upsampling strategies for the decoding process. Then, an adaptive Swin Transformer block and an adaptive upsampling block are built to enable the adaptability for effectively capturing both global and local contexts. Lastly, an evolutionary search algorithm is introduced to systematically explore the search space, identifying the optimal architecture that is highly effective for handling medical image segmentation tasks. Experimental results on benchmark medical image segmentation datasets demonstrate the effectiveness of EW2A, outperforming other competing approaches.