CLEP-Based EEG to Text Reconstruction

With the growing demand for Brain-Computer Interface (BCI) systems among patients with paralysis and aphasia, the challenge of recovering semantic information from language-related EEG signals urgently needs to be addressed, and research on tonal languages remains in its early stages. To address this, we propose a continuous Chinese semantic reconstruction model utilizing EEG signals to enhance system performance, and propose a BCI pre training framework based on the comparison of language and EEG signals. In this framework, CLEP is designed to enhance the accuracy of text reconstruction, while ChatGLM-6B ensures the continuity and coherence of the generated text. The framework was trained on a server using a publicly available Chinese EEG dataset, in addition to data collected from three subjects. The Experimental results were evaluated using two metrics, BERTScore and BLEU. The scores demonstrate that the framework outperforms random baseline tests. In particular, the accuracy of BERTScore score on the self collected dataset has been improved by 0.9%, and the effectiveness of the model has been demonstrated through Z-test. In addition, the feature capture ability of the CLEP model and the authenticity of human brain region responses were verified through feature extraction and visualization experiments.