Unsupervised Anomaly Detection in Deep Learning: From Autoencoders to Transformers

Abstract Anomaly detection is a significant problem in various applications ranging from cyber security to manufacturing quality control, which necessitates the ability to detect rare and unusual patterns. With the development of deep learning from traditional methods, it is possible to learn finer-grained data representation and pattern structure in complex high-dimensional information. We thoroughly review unsupervised deep learning methods for anomaly detection in this work, ranging from classical ones to the latest transformer-based. The contributions of our paper are as follows: 1) We conduct a comprehensive survey on state-of-the-art unsupervised deep models with attention mechanisms; 2) We investigate and summarize the applicability and possible variants of them, covering their pros and cons. We study the subfamilies of autoencoders, such as variational and adversarial autoencorders, discuss GAN-based detection techniques and examine recently introduced transformer architectures targeted for outlier detection. By analyzing applications in cyber security, manufacturing defect detection and fraud detection, we explain how these techniques face practical issues while we provide academics with research directions on this fast-evolving domain. Keywords Anomaly detection, Unsupervised learning, Autoencoders, GANs, Transformers, Deep learning, Outlier detection