Robust Low-Rank Sparse Framework for Video-Based Affective Computing

Video-based Affective Computing (VAC) is central to emotion understanding and emerging consumer electronics applications, yet it often suffers from unstable optimization and representational degradation under complex, non-stationary affective dynamics in the wild. A key challenge is that short-term behavioral fluctuations can convey different affective meanings under different long-term emotional contexts, whereas most existing VAC models entangle heterogeneous factors in a single latent space, leading to component mixing and cross-scale inconsistency. We propose the Low-Rank Sparse Emotion Understanding Framework (LSEF), a plug-and-play framework grounded in the low-rank sparse principle that models affective dynamics as the composition of long-term emotional bases and sparse transient perturbations. Specifically,LSEFconsists of four modules: the Stability Encoding Module (SEM) extracts low-rank bases by emphasizing slow-varying context patterns; the Dynamic Decoupling Module (DDM) isolates sparse transients via temporal routing and orthogonalized relational propagation; and the Consistency Integration Module (CIM) restores multi-scale coherence between stability and reactivity under hierarchical aggregation. We further propose a Rank Aware Optimization (RAO) strategy that adaptively modulates perturbation strength according to rank- and sparsity-sensitive structure to improve training stability and generalization. Experiments on discrete and continuous VAC benchmarks show consistent gains in robustness and dynamic discrimination across general CNN-based backbones, validating hierarchical low-rank sparse modeling as a principled foundation for video affect understanding.