ANIMATE: Unsupervised Attributed Graph Anomaly Detection with Masked Graph Transformers

Attributed graphs have recently emerged as a powerful tool for representing diverse data in numerous real-world systems. Among various applications, unsupervised graph anomaly detection (UGAD) aims to identify abnormal data that significantly deviate from the majority of normal nodes without label annotations. Hence, UGAD can provide crucial assistance in enhancing the reliability of intelligent decision support systems, prediction systems, warning systems and so on. Under the class-imbalanced reality caused by anomaly scarcity, the common paradigm of UGAD focuses on learning a model that primarily captures normal patterns. Existing research usually adopts graph neural networks (GNNs) as backbones. Unfortunately, the local-aggregation property of GNNs impairs the discrimination between normality and their opponents. Besides, these models are restricted with network depth by over-smoothing deficiency, limiting their representation capacity. Recently, graph Transformers (GTs) have shown the potential to capture global information. In light of this, we pioneeringly introduce GTs into UGAD task for the first practice, termed as unsupervised attributed graph anomaly detection with masked graph transformers (ANIMATE). The self-attention mechanism and position embedding of Transformers facilitate us to capture the global receptive field, alleviating the limitations of blind local aggregation while preserving the distinguishable characteristics of abnormalities from a global perspective. Furthermore, we employ masked auto-encoders to reconstruct node features and prompt our model to focus more on learning normal patterns. Additionally, we enhance the performance through a self-paced enhancement scheme specifically for UGAD tasks. Experiments conducted on various real-world benchmark datasets with organic anomalies validate the effectiveness of our proposed method.