Emotion Detection Based on Audio-Visual Using Deep Learning

View at publisher Summarise with VIRAT
Authors: Davin Nicholson, Zener Sukra Lie, Esther Widhi Andangsari
Cited by: 0
References: 16

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Emotional recognition is extremely useful for assessing mental health or analyzing customer behavior. While it's generally easy for people to recognize emotions, in the era of globalization, people are often less sensitive to the emotions displayed by their conversation partners. This research focuses on the development of an audio-visual emotion detection system that analyzes both facial expressions and vocal intonations. The system employs Convolutional Neural Networks (CNN) for processing visual data and Long Short-Term Memory (LSTM) networks for analyzing sequential audio signals. The system contributes to Sustainable Development Goal 3 (Good Health and Well-being) by enabling automated emotion recognition for mental health monitoring and user-aware technologies. The models achieved high validation accuracies for visual input and audio input and pretty high for combined audio-visual input. These results demonstrate the feasibility of multimodal emotion detection using deep learning approaches.

Fields of Study & Subjects

Emotion and Mood Recognition Speech and Audio Processing Sentiment Analysis and Opinion Mining

Publication details

Journal / source
International Conference on Cognitive Systems and Computer Interaction (ICoSCI)
Pages
1–6
Publication date
15 January 2026
Language
EN