Internalizing reasoning via chain-of-thought and retrieval augmentation for multimodal sentiment analysis

Read full text
Authors: Yang Sun, Fengling Li, Tianshi Wang, Guoli Yang, Lei Zhu
Cited by: 1
References: 16

Abstract

• We propose a unified framework adapting large models for precise sentiment tasks. • Hybrid tuning strategy merges explicit reasoning with intuitive discrimination. • Label-balanced retrieval selects diverse paradigms to mitigate context bias. • Contextual anchors enable internalized reasoning without verbose generation. • Extensive experiments demonstrate superior performance with high efficiency. Multimodal Sentiment Analysis (MSA) is critical for interpreting user-generated content, yet it remains challenging because textual and visual evidence is often inconsistent, only weakly grounded to fine-grained targets, and easily affected by noisy contextual cues. While Large Vision-Language Models (LVLMs) offer promising generalization capabilities, they often lack the granularity required for specific sentiment tasks and may suffer from hallucinations and unstable reasoning when multimodal cues are ambiguous or conflicting. To address these limitations, we propose a unified framework termed Internalized Reasoning via Chain-of-Thought and Retrieval Augmentation (IR-CTRA). Specifically, we introduce a dual-stream hybrid tuning strategy that integrates explicit chain-of-thought rationales with implicit in-context learning predictions within a shared parameter space. This approach bridges the gap between logical deduction and intuitive discrimination. Furthermore, to mitigate context-induced polarity bias, we design a label-balanced retrieval mechanism that dynamically selects diverse demonstration paradigms from a self-constructed repository. These paradigms serve as contextual anchors during inference, enabling the model to internalize reasoning processes without relying on verbose generation. Extensive experiments on benchmark datasets demonstrate that IR-CTRA achieves state-of-the-art performance across both sentence-level and aspect-level tasks, exhibiting superior parameter efficiency compared to larger-scale baselines, and achieving an accuracy improvement of up to 16.78% on the Twitter-2015 dataset over comparably sized models. The code and models are available at https://github.com/DandelionWow/IR-CTRA .

Fields of Study & Subjects

Sentiment Analysis and Opinion Mining Emotion and Mood Recognition Topic Modeling

Publication details

Journal / source
Information Fusion
Publisher
Elsevier BV
Volume
135
Pages
104491–104491
Publication date
19 May 2026
Language
EN