Multimodal Fusion of EEG and Physiological Signals for Robust Emotion Recognition via Machine Learning

Abstract Emotion recognition based on physiological activity has become a vital component of affective computing and human–machine interaction. Electroencephalography (EEG) offers direct neural insight into emotional processing but suffers from noise sensitivity, motion artifacts, and high inter-subject variability. Peripheral physiological signals such as electrocardiography (ECG), galvanic skin response (GSR), photoplethysmography (PPG), and respiration provide stable indicators of autonomic changes but lack the specificity necessary for distinguishing subtle emotional states. Multimodal fusion has emerged as a promising solution by integrating complementary bio signals to build more reliable and generalizable affect-recognition models. This study presents a comprehensive review of multimodal EEG–physiology fusion systems, examining fusion strategies, representation-learning approaches, robustness challenges, and cross-subject generalization. Evidence from recent literature shows that unimodal EEG systems typically achieve 60–72% accuracy, while multimodal fusion methods reach 82–90%, and advanced hybrid deep-fusion architectures exceed 92% in subject-independent evaluations. Key limitations persist, including heterogeneous feature spaces, misaligned modalities, sensor noise, and dataset inconsistencies. The synthesis identifies hybrid deep-learning fusion as the most resilient and effective strategy, establishing multimodal physiological learning as a foundational pathway toward future real-time, noise-robust, and generalizable emotion-recognition technologies.