PVT-MSFF: A Pyramid Vision Transformer with Multi-Scale Feature Fusion for polyp segmentation in endoscopic images

Background and ObjectiveColorectal cancer (CRC) is one of the most fatal malignancies worldwide, and its early diagnosis relies on the accurate detection and segmentation of polyps in endoscopic images. However, existing methods are often challenged by the diverse morphology of polyps, ambiguous boundaries, and differences in data centers, which can lead to missed detections and limited generalization.MethodsWe propose a novel Pyramid Vision Transformer with Multi-Scale Feature Fusion network (PVT-MSFF), which combines a Pyramid Vision Transformer encoder with a cooperative multi-module strategy. Our method introduces a Feature Enhancement Module (FEM) with cross-attention mechanism, a Multi-Scale Fusion Module (MSFM) for hierarchical feature expression, and a Global Context Sensing (GCS) module to enhance boundary sensitivity and semantic integration.ResultsWe evaluate the proposed PVT-MSFF on five public polyp segmentation benchmark datasets including Kvasir-SEG, CVC-ClinicDB, CVC-ColonDB, ETIS and CVC-300. Experimental results show that our PVT-MSFF outperforms other state-of-the-art methods across all benchmark datasets. To further validate the practical applicability of our method, we collaborate with experienced gastroenterologists from a leading hospital to construct a new clinical dataset (G-endoscope). Evaluation on this dataset confirms the robust generalization and accuracy of our method.ConclusionThese promising results highlight the potential of PVT-MSFF for broader deployment in clinical scenarios including intelligent endoscopy systems, computer-assisted diagnosis and remote healthcare application. Our code will be available at: https://github.com/jize123457/PVT-MSFF.