How Do LLMs Generate Contrastive Sentiments? A Mechanistic Perspective

Read full text Summarise with VIRAT
Authors: Van Bach Nguyen, Jörg Schlötterer, Christin Seifert
Cited by: 0
References: 20

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

This paper presents a mechanistic investigation of how large language models (LLMs) generate contrastive sentiments.We define this task as transforming the sentiment of a given text (e.g., from positive to negative) while making minimal changes to its content.We identify two core mechanisms: (1) a preservation mechanism that maintains the sentiment of the input text, primarily mediated by specific attention heads, and (2) a sentiment transformation mechanism, which integrates a representation of the target sentiment label with the original valenced words using a circuit containing both MLP and attention layers.Building on these findings, we propose and validate a novel mechanistic intervention.By modifying key attention heads, we steer the LLM toward more effective contrastive generation, increasing the sentiment flip rate without sacrificing the minimality of changes.Our work not only deepens the understanding of the mechanisms underlying contrastive sentiment generation in LLMs, but also introduces a promising new direction to steer LLM behavior via targeted, mechanistic interventions. 1

Fields of Study & Subjects

Sentiment Analysis and Opinion Mining Topic Modeling Explainable Artificial Intelligence (XAI)

Publication details

Journal / source
Conference of the European Chapter of the Association for Computational Linguistics
Pages
6619–6635
Publication date
1 January 2026
Language
EN