BAT: A Bidirectional Adaptive Feature-Aided Transformer for Lightweight Medical Image Segmentation

Accurate delineation of anatomical structures in medical imagery forms the foundation for reliable diagnosis and therapeutic planning. While architectures derived from U-Net have seen predominant adoption, their extensive parameter sizes and demanding compute requirements pose significant barriers to deployment in real-world clinical environments with limited resources. To address these limitations, we propose the Bidirectional Adaptive Feature-Aided Transformer (BAT), a lightweight effective decoder module for medical image segmentation. BAT integrates a Panoptic Block that captures multi-scale spatial features with minimal overhead, and an Adaptive Interaction Attention mechanism that enables bidirectional local-global feature fusion. Applied with a U-shaped encoder-decoder framework, BAT simultaneously enhances contextual understanding and reduces computational complexity. Extensive experiments on four benchmark datasets (ISIC 2018, Kvasir-SEG, CVC-ClinicDB, and CVC-ColonDB) demonstrate that BAT achieves state-of-theart segmentation accuracy (up to +2.4 Dice improvement over strong baselines) while significantly reducing parameter count and FLOPs compared to existing transformer-based models. These results highlight BAT as a promising step toward accurate and resource-efficient medical image segmentation.