Physical-Semantic-Aware Multimodal Facial Expression Recognition for Human-Centric IoT
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
Facial Expression Recognition (FER) serves as a foundational sensory interface for Human-Centric IoT, supporting applications such as smart healthcare monitoring and affective intelligent environments. However, real-world performance is often hindered by theSemantic Gap, where models confuse visually similar expressions that arise from fundamentally different physiological muscle movements. To bridge this gap, we propose the Physical-Semantic-Aware Multimodal Framework (PSM-FER), which introduces 3D Blendshape (BS) coefficients as explicit physical priors to encode high-level muscle motion semantics. Our framework utilizes two synergistic pathways:Direct Physical Gating(DPG) for robust feature modulation andSemantic-Guided Spatial Attention(SGSA) for anatomical spatial recalibration. Additionally, an auxiliary physical regression task enforces anatomical consistency by regularizing the latent features to follow underlying biomechanical laws. Extensive experiments on the RAF-DB dataset demonstrate that PSM-FER achieves an accuracy of 92.37%, establishing a robust and interpretable foundation for affective sensing in complex IoT ecosystems.
Fields of Study & Subjects
Emotion and Mood Recognition
Face recognition and analysis
Face and Expression Recognition
Publication details
- Journal / source
- IEEE Internet of Things Journal
- Volume
- 13
- Issue
- 13
- Pages
- 28153–28164
- Publication date
- 27 March 2026
- Language
- EN