A Comparative Study of Deep Learning Models for Log Anomaly Detection Using the HDFS Dataset

In this paper, we examine log-based anomaly detection based on four deep learning models: Autoencoder, Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and Transformer. We take the Hadoop Distributed File System (HDFS) dataset, train and test all models using an identical preprocessing pipeline to ensure fair comparison. We perform the log parsing, session formation, and sliding-window generation to convert raw logs into sequential representations suitable for learning. Empirical results demonstrate the capacity that LSTM and Transformer can capture long-term dependencies on log sequences, and CNN and Autoencoder have a moderate detection performance, but they are not able to handle rare or complicated abnormal patterns. The results provide the significance of sequential modeling on log anomaly detection, and they also show that contemporary deep learning approaches substantially surpass reconstruction-based or convolution-based counterparts in standardized environments.