A Comprehensive Survey of Multimodal Data Integration for Cognitive Disorders and Behavior Understanding

In this survey, we study the use of multimodal fusion methodology in cognitive state analysis applications, applications in human behaviour modeling, and in the early diagnosis of neurological disorders. The literature review has shown that the combination of various modalities of MRI, PET, EEG, speech, video, and textual data has a considerable higher level of diagnostic accuracy, interpretability, and strength, in comparison to monomodal approaches. More recent algorithms such as graph-based fusion, attention based transformers and multimodal neuroimaging models give more elaborate feature representations and allow prediction of disease risk earlier, especially in the case of Alzheimer and mild cognitive impairment. Even though the performance improvements are realized, issues in availability of datasets, scale validation, imbalance of modalities, and computation overhead remain. The scalable learning, longitudinal monitoring, standardized clinical datasets and privacypreserving multimodal integration should be the focus of future studies.