A Novel Hybrid Deep Learning Model using U-net and Vision Transformer for Polyp Segmentation in medical imaging

This research represents a hybrid model for polyp segmentation in medical images, which combines convolutional neural networks (CNN) and vision transformers (ViT). We propose a hybrid architecture that integrates UNet with a Vision Transformer, where UNet is used as an encoder to capture local features and ViT is used as a bottleneck module to extract global contextual information. To improve the robustness and generalization of the model, various data augmentation techniques are applied to the training data set, including horizontal flip, vertical flip, and rotation. The model is trained and evaluated on the multicenter PolypDB dataset, with all images preprocessed to a consistent resolution, and the corresponding masks are appropriately managed. A loss function, combining Dice Loss and Binary Cross-Entropy (BCE) Loss, is applied to optimize segmentation performance.