A Hybrid Facial Expression Recognition System Based on Machine Learning and Deep Learning Models

Read full text Summarise with VIRAT
Authors: Akkapop Prasompon, Prompong Sugunnasil, Atogorn Sanguansri
Cited by: 0
References: 34

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Pain assessment through facial expressions is an important area of research because many patients cannot clearly communicate their pain levels. This study presents the first systematic comparison of continuous time-series versus tokenized sequence representations for facial action unit (AU)-based pain classification, introducing a novel application of NLP models (BERT) to discretized AU sequences treated as symbolic text. Two datasets were used: the UNBC-McMaster Shoulder Pain Archive (UNBC-SP) with about 48,000 frames, and the Multimodal Intensity Pain (MIntPain) dataset with about 187,900 frames. Action unit intensities were extracted using the Py-Feat library and then normalized, oversampled, and augmented. A range of models was tested, including random forest, support vector machine, recurrent neural networks, and BERT. Key contributions include: (1) demonstrating that continuous time-series models significantly outperform tokenized approaches (91% vs. 82% accuracy); (2) revealing that classical ensemble methods surpass deep learning on tokenized sequences in data-limited scenarios; and (3) establishing that disruptive augmentations harm performance while conservative methods maintain accuracy. The continuous-time series models achieved the best performance, reaching 91% accuracy on MIntPain and 84% on UNBC-SP, while the tokenized models peaked at 82%. The results suggest that preserving temporal details of facial action units provides an advantage for pain detection, especially with larger datasets, though tokenization may retain value in resource-limited settings. The study highlights the need for larger, more diverse datasets and for validation in real clinical settings to improve the reliability of automatic pain recognition.

Fields of Study & Subjects

Emotion and Mood Recognition Face recognition and analysis Face and Expression Recognition

Publication details

Journal / source
ASEAN Journal of Scientific and Technological Reports
Volume
29
Issue
4
Pages
e261424–e261424
Publication date
25 March 2026
Language
EN