A Multi-Modal Entropy-Weighted Fusion Framework for Melanoma Detection

Malignant form of skin melanoma is the most lethal form of skin disease. Therefore, timely detection is extremely crucial for a patient's survival. Traditional methods to detect malignant skin melanoma are often time-consuming and resource intensive. This paper proposes an entropy-weighted attention-based fusion framework, combined with a robust ensemble of classifiers, that classify skin lesions as benign or malignant. We take advantage of multiple pre-trained models, namely Vision Transformer (ViT), ResNet-50, and MobileNetV3, for potent feature extraction from dermoscopic images. The features extracted using the mentioned deep learning models are then normalized, projected, and concatenated with weights. The concatenated matrix of features is then used to train the ensemble. Experimental results show an overall accuracy of 91.2%. This study highlights the potential of blending feature extraction and attention-based fusion with ensemble classification for effective melanoma detection.