A neuromorphic hybrid spiking-CNN model for emotion recognition in low-resource Kannada speech

Abstract Accurately recognizing human emotions from speech is becoming increasingly important for advancing intelligent and adaptive technologies. Yet, many existing Speech Emotion Recognition (SER) models continue to struggle with suboptimal accuracy, limiting their adoption in practical settings. A key challenge lies in the fact that emotional cues in speech are often subtle, irregular, and represented as weak temporal signals that are easily masked within conventional audio features. In this research, we present a dual-path SER architecture tailored for Kannada speech, combining a Convolutional Neural Network (CNN) to capture spectral–spatial representations with a Spiking Neural Network (SNN) enhanced by a Perceptual Neuron Encoding Layer (PNEL) to model fine-grained temporal pulse patterns. The system is trained and evaluated on the Kannada Emotional Speech Dataset (KESD), comprising acted samples of six emotions: happiness, anger, sadness, fear, surprise, and neutral. PNEL transforms raw audio into spike sequences for a Leaky-Integrate-and-Fire SNN, which complements the CNN branch. Experimental analysis across varied segment lengths, hop sizes, and learning rates shows that the CNN+SNN fusion reaches an accuracy of 65.3% on KESD—surpassing the best baseline (64.7%)—while maintaining a feasible compute budget (∼10.5 h training and ∼10.3 GB memory usage). When subjected to narrow-band noise, performance decreases to 57.7%, indicating further scope for robustness improvements. These results establish a foundation for scalable and noise-resilient Kannada SER, with potential extensions to real-time and cross-lingual emotion recognition.