A hybrid model for improved nail disease classification using vision transformers and stable diffusion

Nail diseases, including various fungal infections, paronychia, and psoriasis, affect millions, and their treatment relies on visual inspection, which may result in inaccurate and delayed treatments. This study explores the use of synthetic nail disease data generated through stable diffusion models to increase the accuracy of the machine learning model. We aim to incorporate enhanced data transformation techniques by utilizing a few-shot learning technique with a small amount of data representation in the text-to-image stable diffusion model. This model renders synthetic data, which provides variety and diversity while preserving the critical components of the original data. The experimental results display the efficacy of the synthetic data that helps provide robustness to the pre-trained CNN MobileNetV2 and Vision Transformer model on a custom real-world nail disease dataset, providing an increased accuracy of 3.26% for MobileNetV2 and 3.02% for Vision Transformers.