Lightweight-oriented classification using distillation learning and spatial information from tabular data

Spatial information, along with deep learning approaches, plays a crucial role in classification tasks. However, tabular datasets do not inherently contain spatial information, so researchers often focus primarily on traditional machine learning algorithms or DNNs for simplicity. Thus, these methods cannot fully utilize all the information in tabular data because they are unable to mine spatial (2D) information. Additionally, while deep learning, particularly convolutional neural networks (CNNs), can handle spatial information effectively, it requires significant computational resources to achieve high performance. In this paper, we present a new method, named ESATTO (Efficient Synthetic Spatial Awareness for Tabular daTa classificatiOn), which uses distillation learning and is able to extract spatial features in an efficient way. The proposed model is 27 times smaller in size and has 50 times fewer parameters than standard models, making it highly efficient for resource-limited environments. This lightweight model is particularly beneficial to real-world applications, such as precision agriculture, where computational resources are constrained. The validation shows that our model outperforms in wine quality prediction and performs strongly on medical data, highlighting its versatility for various tasks. The code is available at https://github.com/tmtgssi/ESATTO .