Controllable Diffusion-Based Skin Lesion Synthesis for Enhanced Segmentation on ISIC2018

Diffusion-based generative models have become a dominant force in medical image analysis, mainly due to their capacity of generating high-quality synthetic samples that compensate for the commonly observed scarcity of annotated data. By using controllable diffusion models, ControlPolypNet generates colon polyp images of interest. Motivated by this paradigm, we have adopted the technique of dermatology for skin lesion segmentation. We have adapted ControlPolypNet’s controlled diffusion pipeline for ISIC2018 benchmark where it is possible to generate dermoscopic images conditioned on lesion masks. The synthesised images were mixed with real dermoscopic images to form augmented training sets, utilised for three segmentation backbone networks: U-Net, TransUNet, and UDTransNet. Quantitative experiments results show that when the ratio achieves 2:1 of synthetic-to-real image data, a salient improvement is achieved for segmentation performance on the enlarged training set. The F1 score is improved by around 5% for U-Net, 4% for TransUNet, and 2% for UDTransNet over the models only trained on original images. These results demonstrate that controlled diffusion-based synthesis is an efficient approach to enhance skin lesion segmentation. While its full application potential is not yet exhausted, the findings indicate that this method could be of great importance for digital processing of dermatological images.