Hyperbolic Convolutional Knowledge Representation with Attention Mechanism for Multiple Patterns

Most of existing knowledge representation approaches are based on the Euclidean geometry with limited representation capabilities while the hyperbolic space comes with stronger expressivity and natural hierarchical property. In addition, there are various relation patterns in knowledge graphs and different geometric operations possess different learning strengths for the patterns. Therefore, a hyperbolic knowledge representation framework is proposed. Firstly, lowdimensional fact embeddings are learned in the Poincaré ball space while the problem of vanishing gradients in previous hyperbolic counterparts is solved. Secondly, relation patterns are modeled with various geometric operations and features are integrated with a hyperbolic attention mechanism. Last but not least, a hyperbolic convolution layer is introduced to improve the feature extraction capability. Such framework promotes sustainable AI practices by enabling compact low-dimensional representations with high expressivity, reducing both memory footprint and computational demand. Link prediction is conducted on benchmark datasets comparing our method with state-of-the-art models, the latter improving by 6.76% at Hits@3 on YAGO3-10 compared to HYBONET. An ablation study is also carried out to verify the effectiveness of each component.