Collaboration of Fusion and Independence: Hypercomplex-driven Robust Multi-Modal Knowledge Graph Completion

Read full text
Authors: Zhiqiang Liu, Yichi Zhang, Mengshu Sun, Lei Liang, Wen Zhang
Cited by: 0

Abstract

Multi-modal knowledge graph completion (MMKGC) aims to discover missing facts in multi-modal knowledge graphs (MMKGs) by leveraging both structural relationships and diverse modality information of entities.Existing MMKGC methods follow two multi-modal paradigms: fusion-based and ensemble-based.Fusion-based methods employ fixed fusion strategies, which inevitably leads to the loss of modality-specific information and a lack of flexibility to adapt to varying modality relevance across contexts.In contrast, ensemble-based methods retain modality independence through dedicated sub-models but struggle to capture the nuanced, context-dependent semantic interplay between modalities.To overcome these dual limitations, we propose a novel MMKGC method M-Hyper, which achieves the coexistence and collaboration of fused and independent modality representations.Our method integrates the strengths of both paradigms, enabling effective cross-modal interactions while maintaining modality-specific information.Inspired by "quaternion" algebra, we utilize its four orthogonal bases to represent multiple independent modalities and employ the Hamilton product to efficiently model pair-wise interactions among them.Specifically, we introduce a Fine-grained Entity Representation Factorization (FERF) module and a Robust Relation-aware Modality Fusion (R2MF) module to obtain robust representations for three independent modalities and one fused modality.The resulting four modality representations are then mapped to the four orthogonal bases of a biquaternion for comprehensive modality interaction.Extensive experiments indicate its state-of-the-art performance with better robustness.Our dataset and code are available at https://github.com/zjukg/M-Hyper.

Fields of Study & Subjects

Semantic Web and Ontologies Service-Oriented Architecture and Web Services Cognitive Computing and Networks

Publication details

Pages
27968–27983
Publication date
1 January 2026
Language
EN