Multimodal Emotion Recognition Using Deep Learning with Voice, Text, and Facial Expression Analysis

Read full text Summarise with VIRAT
Authors: P. Praveenkumar, S. Yogithaa, R. Soundarya, M. Harshini
Cited by: 0
References: 1

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Emotion recognition plays a crucial role in intelligent systems, as emotions influence communication, decision-making, and human-machine interaction.Audio-only methods such as CNN-BiLSTM often perform poorly because emotional expression varies across speech, facial cues, and textual semantics.This study proposes a multimodal framework integrating text, audio, and facial expressions for robust emotion detection.Text is modeled with BiLSTM to capture contextual meaning, audio is processed through a CNN-RNN hybrid to learn spectral-temporal cues, and visual data is analyzed using ResNet-101 for deep facial feature extraction.Feature-level fusion combines all modalities into a unified emotional representation, improving accuracy and stability across real-world conditions.The approach benefits applications in HCI, e-learning, affective computing, and mental-health monitoring.

Fields of Study & Subjects

Emotion and Mood Recognition Sentiment Analysis and Opinion Mining Speech and Audio Processing

Publication details

Journal / source
Advances in intelligent systems research/Advances in Intelligent Systems Research
Pages
249–261
Publication date
1 April 2026
Language
EN
Document type
Book chapter