FED: A Fine-Grained Enhanced Dual-Routing Network for Multimodal Sarcasm Detection

Multimodal sarcasm detection aims to detect sarcastic emotions within multimodal data by integrating explicit modal features and implicit emotional information. Prevailing studies directly reverse textual representations using a universal prompt to provide antipodal perspectives for mitigating spurious correlations. However, the reliance on low-quality data generated from the single prompt inevitably introduces additional noise. To this end, we propose a novel framework, Fine-grained Enhanced Dual-routing (FED) for the detection of multimodal sarcasm. Specifically, we design a hierarchically modality-aware mechanism to identify vital sarcastic entities and sarcastic opinions from textual and visual modalities, and leverage semantic decoupled prompts to generate high-quality antonymic text samples. Then, the dynamic semantic dual-routing module is employed to select the optimal paths for adaptively capturing the complex semantic relations between text and images. Extensive experimental results on the public dataset demonstrate the superior performance of our method compared with baselines.