Facial Emotion Recognition with Attention-Driven Deep Learning: A Lightweight Custom Architecture and Ensemble-Based Framework

Facial Emotion Recognition (FER) has been identified as a key element in the development of modern intelligent systems in various applications such health, education, surveillance and human–computer interaction. Nevertheless, currently available FER models generally suffer from high computation cost, dataset bias and poor generalization ability. In this paper, we propose a new FER framework with the solution to the problems using custom-based Convolutional Block Attention Module (CBAM) model and deep architecture ensembles. To this end, the proposed FaceNet-CBAM model adopts a lightweight dual path attention mechanism, i.e., channel and spatial attention to highlight emotion-related facial regions by suppressing irrelevant features. Similarly to usual pretrained backbone models consisting of millions parameters, FaceNet-CBAM was trained from scratch. Experimental results show that the FaceNet-CBAM model could attain 95.82% Accuracy on RAF-DB dataset while the ensemble model, using FaceNet-CBAM, DenseNet201 and MobileNetV3 could get 98.22%, outperforming current high-performance approaches. The proposed architecture presents an optimal trade-off between the performance and hardware cost, thus being well-suited for real-time FER usages.