A Multimodal Framework for Aphasia Severity Classification in Russian

Read full text Summarise with VIRAT
Authors: Anastasia Vladimirovna Kolmogorova, Ekaterina Yavshitz, Anastasia V. Margolina, Anna Sugyan
Cited by: 0
References: 23

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Automatic classification of aphasia severity presents persistent challenges, particularly for languages with limited clinical speech resources such as Russian.This paper explores a multimodal approach to severity estimation that combines acoustic and semantic representations of pathological speech.Acoustic features are extracted using pretrained Wav2Vec 2.0 models, while semantic information is obtained from the encoder of the Whisper model.The two representations are integrated via early feature fusion and evaluated using gradient boosting classifiers in a speaker-independent crossvalidation setting.Experiments are conducted on a newly collected dataset of Russian speech recordings from patients with aphasia and neurotypical speakers (RuAphasiaBank).The results suggest that the combined use of acoustic and semantic embeddings can provide more stable severity estimates than unimodal baselines.This study contributes empirical evidence on the applicability of multimodal representation learning for aphasia severity classification under data-scarce conditions.

Fields of Study & Subjects

Speech Recognition and Synthesis Voice and Speech Disorders Neurobiology of Language and Bilingualism

Publication details

Journal / source
Workshop on Linguistic Analysis for Health (HeaLing)
Pages
257–265
Publication date
1 January 2026
Language
EN