Multimodal Emotional Art Generation

View at publisher Summarise with VIRAT
Authors: Tamil Thendral M, Jampani Likhitha, Jakka Venkata Jyothsna
Cited by: 0
References: 14

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Conventional music recommendation systems primarily rely on historical user preferences and do not adapt to users’ real-time emotional states, resulting in limited personalization. This paper proposes a multimodal emotion aware music recommender system that uses various deep learning models for emotion classification. The system uses a fine-tuned VGGFace network for facial emotion recognition based on the Facial Expression Recognition 2013 (FER2013) database. For speech emotion recognition, it uses a fine-tuned Long Short-Term Memory (LSTM) network based on the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) database. It uses a fine-tuned Bidirectional Encoder Representations from Transformers (BERT) network for textual emotion classification based on the GoEmotions database. Our experiments revealed that the multimodal emotion classification system obtained a superior emotion recognition accuracy of 86.9% when compared to unimodal emotion classification systems. The class of emotion recognized by the system is then used to generate music based on attributes like tempo, rhythm, valence, arousal, etc. The user evaluation of our system proved to be more satisfying compared to other systems.

Fields of Study & Subjects

Emotion and Mood Recognition Recommender Systems and Techniques Music and Audio Processing

Publication details

Journal / source
International Conference on Computing, Communication, Control and Cyber-Physical Systems (I5CPS)
Pages
1–6
Publication date
26 February 2026
Language
EN