An Empirical Study Challenging the Ability of Heterogeneous Graph Neural Networks to Effectively Learn Topological Structures

The utilization of graph neural networks in heterogeneous networks has been rapidly increasing in recent years. The topological structure, fundamental to the nature of graphs, is often claimed to be jointly addressed with attribute information by most heterogeneous graph neural network (HGNN) approaches. However, this assertion lacks sufficient empirical validation. We empirically challenge these claims within existing evaluation frameworks. We propose a straightforward multi-layer perceptron (MLP) based model which relies solely on adjacency relationships for feature aggregation, without fully exploiting the graph's intricate topological structures. Unexpectedly, the model achieved nearly state-of-the-art performance on several real-world datasets. This discrepancy indicates that current evaluation protocols contain systemic biases and often fail to distinguish superficial adjacency exploitation from genuine structural pattern learning. It also points to the need for future benchmark datasets and metrics that more faithfully reflect the real requirements of topological learning. We hope this work stimulates further research into the role and significance of topological information in HGNNs.