BERT WEIGHT: Word-Level Sentiment Analysis through Contextual Token Attribution and Embedding Weighting

View at publisher Summarise with VIRAT
Authors: Raunit Jain, Khilan Kanadia, Raaj Shah, Jinay Jain, Kranti Vithal Ghag, Meera Narvekar
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

This paper introduces BERT WEIGHT, a novel framework that addresses the interpretability limitations of transformer-based sentiment analysis by leveraging word-level contextual information. Unlike conventional approaches that rely solely on the [CLS] token or pooled representations, our methodology assigns sentiment weights to individual tokens, providing fine-grained analysis and enhanced transparency. We propose three distinct methods: (1) an attention-based token attribution approach that utilizes self-attention log-ratios from top layers to identify class-specific token contributions, (2) a word-level average embedding technique that computes sentiment scores across all 12 BERT layers, and (3) a CLS embedding norm-based scoring method that quantifies each term’s influence through systematic evaluation of embedding patterns. Extensive experiments on Amazon, IMDB, and Yelp datasets demonstrate that our approach not only outperforms traditional sentiment models but also provides explicit mapping between sentiment predictions and specific words in the text. By moving beyond opaque sentence-level classifications to transparent word-level sentiment attribution, BERT WEIGHT offers a significant advancement for applications where explainability is essential while maintaining competitive performance.

Fields of Study & Subjects

Sentiment Analysis and Opinion Mining Explainable Artificial Intelligence (XAI) Topic Modeling

Publication details

Journal / source
CRC Press eBooks
Pages
271–274
Publication date
25 March 2026
Language
EN
Document type
Book chapter