MRBiLSTM: Integrating Relative Position-Aware Multi-Head Attention with Bidirectional LSTMs for Fine-Grained Sentiment Analysis
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
Sentiment analysis remains a critical challenge in natural language processing due to the complexity of capturing contextual dependencies and sentiment-bearing linguistic patterns. This paper presents MRBiLSTM (Multi-Head Relative Attention with Bidirectional LSTM), a novel hybrid architecture that synergistically integrates bidirectional Long Short-Term Memory networks with multi-head relative position-aware attention mechanisms for fine-grained sentiment classification. Unlike conventional approaches using absolute positional encodings, our model explicitly captures pairwise token distances through relative position embeddings, enabling translation-invariant pattern recognition across variable-length sequences. Evaluated on the IMDB movie review dateset containing$\mathbf{5 0, 0 0 0}$samples, the proposed architecture achieves 88.56% accuracy, 88.80% F1-score, 0.9518 AUC-ROC, and exceptionally low calibration error as$\mathbf{E C E} \boldsymbol{=} \mathbf{0. 1 0 0 8}$. Comprehensive ablation studies demonstrate the effectiveness of our regularization strategies including dropout, layer normalization, gradient clipping, and early stopping. Attention weight visualizations reveal interpretable decision-making processes, with the model successfully identifying sentiment-critical tokens. The results validate that combining recurrent sequential modeling with relative attention mechanisms produces robust, well-calibrated, and interpretable sentiment classifiers suitable for production deployment in sentiment-driven applications.
Fields of Study & Subjects
Sentiment Analysis and Opinion Mining
Topic Modeling
Stock Market Forecasting Methods
Publication details
- Journal / source
- International Conference on Intelligent Sustainable Systems (ICISS)
- Pages
- 1489–1494
- Publication date
- 4 March 2026
- Language
- EN