ICoCO: Interpretable Concept-Guided Context Optimization for Trustworthy Facial Expression Recognition in Mental Health Monitoring
Abstract
Facial expression recognition (FER) plays a crucial role in healthcare, particularly in mental health monitoring, yet it remains challenging due to the subtle and ambiguous nature of expressions. Existing language-guided FER methods primarily rely on global image–prompt similarity. However, they often overlook fine-grained local cues, which weakens visual–semantic alignment and hinders cross-dataset generalization. To address these limitations, we proposeInterpretableConcept-guidedContextOptimization (ICoCO), a novel framework that captures subtle facial variations and achieves multi-granularity visual–semantic alignment. Specifically, we design a multi-granularity Emotion-aware Visual Adapter (MEVA) to model subtle facial differences while improving intra-class compactness and inter-class separability of emotional appearance embeddings. In addition, we introduce an Emotion Concept-guided Context Optimization (ECoOp) module, which distills fine-grained emotion concepts from large language models into learnable prompts and establishes concept–appearance coupling between vision and language, thereby enabling more precise and comprehensive understanding of expressions. Furthermore, we formulate FER as an optimal transport problem between visual and semantic distributions, achieving effective local-to-global alignment for interpretable expression analysis. Extensive experiments on RAF-DB, AffectNet, ExpW, and FERPlus demonstrate that ICoCO not only achieves state-of-the-art performance but also exhibits robust occlusion handling, strong cross-dataset generalization, and interpretable analysis, highlighting its potential for real-world mental health applications.
Fields of Study & Subjects
Emotion and Mood Recognition
Face recognition and analysis
Face and Expression Recognition
Publication details
- Journal / source
- IEEE Transactions on Affective Computing
- Pages
- 1–13
- Publication date
- 1 January 2026
- Language
- EN