Physical-Semantic-Aware Multimodal Facial Expression Recognition for Human-Centric IoT

View at publisher Summarise with VIRAT
Authors: Ke Wang, Dongxu Li, Xin Xie, Xiaoyi Tao, Xiulong Liu, Zheng Gong, Sheng Chen, Keqiu Li
Cited by: 0
References: 37

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Facial Expression Recognition (FER) serves as a foundational sensory interface for Human-Centric IoT, supporting applications such as smart healthcare monitoring and affective intelligent environments. However, real-world performance is often hindered by theSemantic Gap, where models confuse visually similar expressions that arise from fundamentally different physiological muscle movements. To bridge this gap, we propose the Physical-Semantic-Aware Multimodal Framework (PSM-FER), which introduces 3D Blendshape (BS) coefficients as explicit physical priors to encode high-level muscle motion semantics. Our framework utilizes two synergistic pathways:Direct Physical Gating(DPG) for robust feature modulation andSemantic-Guided Spatial Attention(SGSA) for anatomical spatial recalibration. Additionally, an auxiliary physical regression task enforces anatomical consistency by regularizing the latent features to follow underlying biomechanical laws. Extensive experiments on the RAF-DB dataset demonstrate that PSM-FER achieves an accuracy of 92.37%, establishing a robust and interpretable foundation for affective sensing in complex IoT ecosystems.

Fields of Study & Subjects

Emotion and Mood Recognition Face recognition and analysis Face and Expression Recognition

Publication details

Journal / source
IEEE Internet of Things Journal
Volume
13
Issue
13
Pages
28153–28164
Publication date
27 March 2026
Language
EN