3-D Relational Learning in Tabular Data: A Dual-Graph Attention Approach

Tree-based ensemble models perform well on tabular data, but their assumption of feature independence restricts their ability to capture the complex relationships between features and samples. To overcome this limitation, several deep learning based approaches have been proposed. However, most prior studies have focused on a single relational axis, either the relations between features or between samples. While some recent efforts have attempted to incorporate both axes jointly, research that integrates feature interactions within a sample, along with the existing relational axes, remains limited. In this study, we propose a framework that utilizes three types of relational information in tabular data: relations between features, relations between samples, and interactions among features within each sample. We note that the importance of the features can vary across samples. As such, our model integrates both unsupervised, structure-aware feature importance, estimated through global inter-feature relations, and Attention-based sample representations derived from the FT-Transformer encoder. The proposed architecture consists of three components: (i) A Feature Graph-based tournament that identifies the most informative numerical features using global feature relations. (ii) An FT-Transformer encoder that captures within-sample feature interactions. (iii) A Sample Graph that models structural relations between samples. Experiments on 11 benchmark datasets demonstrate that our model achieves a 1.3% improvement in average AUC-ROC compared to gradient boosted decision trees (GBDT) and a 6.2% improvement over representative deep learning models. Regarding efficiency, our framework demonstrates a shorter average training time compared to TabNet and SAINT on the same datasets. Furthermore, its computational complexity scales approximately linearly with the number of samples and selected features, making it well-suited for medium-scale tabular datasets.