Collaboration of Fusion and Independence: Hypercomplex-driven Robust Multi-Modal Knowledge Graph Completion
Abstract
Multi-modal knowledge graph completion (MMKGC) aims to discover missing facts in multi-modal knowledge graphs (MMKGs) by leveraging both structural relationships and diverse modality information of entities.Existing MMKGC methods follow two multi-modal paradigms: fusion-based and ensemble-based.Fusion-based methods employ fixed fusion strategies, which inevitably leads to the loss of modality-specific information and a lack of flexibility to adapt to varying modality relevance across contexts.In contrast, ensemble-based methods retain modality independence through dedicated sub-models but struggle to capture the nuanced, context-dependent semantic interplay between modalities.To overcome these dual limitations, we propose a novel MMKGC method M-Hyper, which achieves the coexistence and collaboration of fused and independent modality representations.Our method integrates the strengths of both paradigms, enabling effective cross-modal interactions while maintaining modality-specific information.Inspired by "quaternion" algebra, we utilize its four orthogonal bases to represent multiple independent modalities and employ the Hamilton product to efficiently model pair-wise interactions among them.Specifically, we introduce a Fine-grained Entity Representation Factorization (FERF) module and a Robust Relation-aware Modality Fusion (R2MF) module to obtain robust representations for three independent modalities and one fused modality.The resulting four modality representations are then mapped to the four orthogonal bases of a biquaternion for comprehensive modality interaction.Extensive experiments indicate its state-of-the-art performance with better robustness.Our dataset and code are available at https://github.com/zjukg/M-Hyper.
Fields of Study & Subjects
Semantic Web and Ontologies
Service-Oriented Architecture and Web Services
Cognitive Computing and Networks
Publication details
- Pages
- 27968–27983
- Publication date
- 1 January 2026
- Language
- EN