Energy demand forecasting using deep models and autoencoder- transformer

This study evaluates five prominent deep learning models—CNN-LSTM, Bidirectional LSTM, GRU, Transformer, and the proposed Deep Autoencoder-Transformer for the task of energy demand forecasting. Accurate prediction of energy demand is essential for optimizing consumption and maintaining power grid stability amidst increasing complexity and multivariate data characteristics. While previous research has predominantly assessed more traditional models such as LSTM and GRU, this research fills an important gap by thoroughly comparing these with the Transformer and a novel hybrid autoencoder-Transformer model. The models were systematically trained on multivariate inputs after comprehensive preprocessing and evaluated using statistical metrics including MAE, RMSE, MAPE, and coefficient of determination (R2). The findings demonstrate that the Deep Autoencoder-Transformer model outperforms all other architectures, achieving the lowest error rates (MAE = 8.5, RMSE = 10.75, MAPE = 3.46%) and highest explanatory power (R2 = 0.991). The Transformer also achieves strong performance (MAE = 10.14, R2 = 0.988), reflecting its ability to model long-term dependencies effectively. GRU and Bidirectional LSTM models follow, balancing accuracy and computational efficiency, while CNN-LSTM, despite its combined spatial and temporal feature extraction abilities, shows comparatively lower precision likely due to architectural limitations with long-range temporal modeling. This study highlights the superior capability of attention-based Transformer architectures, especially when combined with deep autoencoding, to capture complex temporal patterns in multivariate energy data. It offers a scalable and systematic framework for benchmarking deep learning models applicable to energy demand forecasting. These insights are valuable to energy system operators and policymakers for selecting appropriate machine learning models, with the hybrid Deep Autoencoder-Transformer emerging as a promising solution for more accurate, long-horizon, multi-step forecasting in intelligent energy systems.