Facial Expression Recognition in Real Time with Domain Adaptive Transfer learning and Feature Integrated CNN

Abstract Automatic facial expression recognition (AFER) in the wild environment is a challenging task in computer vision. However, the advancement of convolution neural networks (CNNs) has improved the detection rate of AFER, but it requires large datasets. Further, CNN based method demands higher computation cost and memory. In this regard, an integrated CNN based model has been proposed with fewer parameters for deployment in real-world settings. The model utilizes domain adaptive transfer learning by modifying VGG-Face to compensate for the small dataset size. Then combine it with a lightweight CNN structure that boosts feature propagation with dense connections. In addition, it uses joint supervision with softmax and center loss to learn the discriminative features of facial expression. Evaluations of four publicly available datasets have been conducted, namely FER-2013, JAFFE, CK + , and KDEF. The comparative result reveals that the proposed method outperforms several existing literature. Finally, to evaluate the effectiveness of the proposed method, an application has been developed with an Intel Core i7 2.9 GHz CPU that detects real-time facial expressions in approximately 55.91 ms/frame.