STBB-Net: A Hybrid EEG Model for Virtual Reality Motion Sickness Detection

Virtual Reality Motion Sickness (VRMS) is a common discomfort reaction when users are immersed in a VR experience, which seriously affects the user experience and limits the widespread application of VR technology. This paper proposes a novel multi-branch hybrid model named STBB-Net for decoding Electroencephalography (EEG) to detect VRMS. The model has three branches including the feature extractor in the ShallowConvNet for local temporal feature extraction, the Transformer encoder to model the global temporal dependency, and the customized brain network module to characterize the topological characteristics among the brain regions. At the same time, a Bayesian Neural Network (BNN) is introduced to fuse the EEG features extracted by the three branches and realize classification as a classifier. BNN introduces the estimation of model uncertainty and leads to improved generalization ability. This paper analyzes resting-state VRMS-related EEG data based on a VR flight obstacle avoidance experiment dataset. The ten-fold cross-validation results demonstrate that the STBB-Net outperforms existing methods across key classification metrics, achieving an accuracy of 97.62% ± 0.52% and an AUC of 99.62% ± 0.19%, highlighting its significant potential for personalized VRMS detection.