Rapid recognition of multimodal emotion based on graph convolutional neural network

In order to solve the problems of high accuracy and recall, as well as high cross entropy loss in traditional methods, a recognition method of multimodal emotion based on graph convolutional neural network is proposed. Utilise independent component analysis algorithm to achieve signal pre-processing, calculate differential entropy features of EEG and ECG signals, and screen key signal features. Using graph convolutional neural networks to extract frequency domain features of signals, incorporating attention mechanisms into bidirectional long short-term memory networks to extract time domain features, and fusing the extracted time domain and frequency domain features to form a comprehensive feature vector. Using Dempster Shafer evidence theory based on feature vectors to determine classifier results and achieve rapid recognition of multimodal emotions. Experimental results have shown that the accuracy and recall of our method consistently remain above 93%, with a cross entropy loss of only 0.05.