Posture-Movement-Frequency-Enhanced Graph Convolutional Network for Gait Emotion Recognition

Recent progress in recognizing emotions through gait analysis has attracted substantial interest. Spatial temporal graph convolutional networks (ST-GCN) have been applied to extract gait features effectively, enabling enhanced emotion recognition. However, existing methods do not account for subtle movement cues that are intricately linked to human emotions, resulting in a lack of representation of emotion intensity. Additionally, these methods fail to consider the cyclic nature of gait, focusing only on local dependencies in the temporal domain. To tackle these limitations, we propose an innovative three-streams graph neural network model PMF-GCN (Posture-Movement-Frequency-enhanced Graph Convolutional Network). First, we introduce for the first time in gait emotion recognition, the integration of movement features from translational and rotational perspectives, combined with frequency-domain data, to capture emotion intensity and global dependencies. Then, we devise a novel adaptive feature fusion mechanism (TR-AFM), which achieves effective extraction of spatial-temporal-specific emotion features from gaits through temporal and spatial attention mechanisms, as well as gated units. Comprehensive experiments on two public datasets show that PMF-GCN achieves leading performance in gait emotion recognition, and achieves state-of-the-art performance.