Design and development of hybrid deep learning technique for deep fake identification: in limited dataset perspective

Abstract Deepfakes are artificial facial images created with the help of AI. They represent a significant threat to web security, online privacy, and the validity of information on the web. Various deep learning models are available for identifying deepfakes. Pretrained CNN models like Inception netv3, Resnet 50, Resnet 101 and DenseNet121 are trained on available benchmark datasets like real and fake face detection with 2k samples, real vs fake face detection with 140k samples and deep fake and real images with 190k samples. The performance of these pre-trained CNN models varies depending on the dataset size and the resolution of the samples. Existing pretrained models have limited performance, i.e. 55% accuracy in the case of imbalanced and limited datasets, like real and fake datasets with 2k samples. In a real-world scenario, it is difficult to get the required number of samples for training and identification of deep fakes. Hence, there is a requirement for a hybrid model that improves the accuracy of deep fake identification from a limited dataset perspective. This paper presents the design and development of a framework that merges GAN with a pretrained model to overcome the challenges of dataset bias. In this paper, GAN is used for the extraction of features of images and to generate new sample images to overcome challenges from a limited dataset perspective. This paper implements a hybrid DL model that merges the capacity of a pretrained model and a GAN with an SE attention mechanism and demonstrates an accuracy of 89%. The proposed hybrid deep learning model with SE attention outperforms pretrained models from a limited dataset perspective.