MRBiLSTM: Integrating Relative Position-Aware Multi-Head Attention with Bidirectional LSTMs for Fine-Grained Sentiment Analysis

View at publisher Summarise with VIRAT
Authors: Battu Chiranjeevi, Rajanala Vijaya Prakash
Cited by: 0
References: 21

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Sentiment analysis remains a critical challenge in natural language processing due to the complexity of capturing contextual dependencies and sentiment-bearing linguistic patterns. This paper presents MRBiLSTM (Multi-Head Relative Attention with Bidirectional LSTM), a novel hybrid architecture that synergistically integrates bidirectional Long Short-Term Memory networks with multi-head relative position-aware attention mechanisms for fine-grained sentiment classification. Unlike conventional approaches using absolute positional encodings, our model explicitly captures pairwise token distances through relative position embeddings, enabling translation-invariant pattern recognition across variable-length sequences. Evaluated on the IMDB movie review dateset containing$\mathbf{5 0, 0 0 0}$samples, the proposed architecture achieves 88.56% accuracy, 88.80% F1-score, 0.9518 AUC-ROC, and exceptionally low calibration error as$\mathbf{E C E} \boldsymbol{=} \mathbf{0. 1 0 0 8}$. Comprehensive ablation studies demonstrate the effectiveness of our regularization strategies including dropout, layer normalization, gradient clipping, and early stopping. Attention weight visualizations reveal interpretable decision-making processes, with the model successfully identifying sentiment-critical tokens. The results validate that combining recurrent sequential modeling with relative attention mechanisms produces robust, well-calibrated, and interpretable sentiment classifiers suitable for production deployment in sentiment-driven applications.

Fields of Study & Subjects

Sentiment Analysis and Opinion Mining Topic Modeling Stock Market Forecasting Methods

Publication details

Journal / source
International Conference on Intelligent Sustainable Systems (ICISS)
Pages
1489–1494
Publication date
4 March 2026
Language
EN