A Knowledge-Graph–Driven Multimodal Large Model for Semantic Understanding and Controllable Generation of Intangible Cultural Heritage

Read full text
Authors: Jundi Yang, Heng Yao
Cited by: 1

Abstract

Intangible Cultural Heritage (ICH) encompasses complex layers of symbolic meaning expressed through motifs, crafts, rituals, and regional traditions.Contemporary multimodal generative models frequently overlook such domain-specific semantics, leading to visually appealing but culturally inaccurate outputs.To address this limitation, this paper introduces a unified knowledge-graph-driven multimodal generation framework that couples a structured ICH Knowledge Graph (KG), a domain-adapted Large Language Model (LLM), and a controllable diffusion-based text-to-image generator.The KG organizes motifs, techniques, symbolic associations, and regional contexts into a structured semantic space, which the LLM leverages to interpret user queries and retrieve culturally grounded constraints.These constraints are injected into the diffusion model through a multi-stage semantic fusion mechanism, enabling culturally faithful and controllable image synthesis.Experimental results across three curated ICH datasets demonstrate that the proposed framework outperforms representative baselines in cultural semantic accuracy, text-image alignment, and robustness to linguistic variation.The proposed approach provides a principled pathway for integrating symbolic cultural knowledge with modern generative models, supporting large-scale preservation, computational interpretation, and creative revitalization of intangible cultural heritage.

Fields of Study & Subjects

Semantic Web and Ontologies Advanced Graph Neural Networks Image Processing and 3D Reconstruction

Publication details

Journal / source
International Journal of Computer Applications
Publisher
Foundation of Computer Science
Volume
187
Issue
77
Pages
1–8
Publication date
31 January 2026