Performance Comparison of Neural Networks Training Algorithms in Time Series Forecasting

Research gaps exist in the area of forecasting since there has been no comparison between different algorithms for training the Feed Forward Neural Network (FFNN) model. One of the main reasons for this existing gap is that it’s hard to decide which training algorithm to choose. This study proposes to fill that gap by identifying the best method to train the FFNN (both shallow and deep learning), a technique for a univariate monthly time series forecast. A total of seven widely known techniques have been studied to compare their efficiency: Quick Propagation Algorithm, Conjugate Gradient Descent Algorithm, Quasi-Newton Algorithm, Limited Memory Quasi-Newton Algorithm, Levenberg Marquardt Algorithm, Online Back Propagation Algorithm (Stochastic Gradient Descent), and lastly the Batch Back Propagation Algorithm. The study was carried out using Alyuda NeuroIntelligence 2.2. Additionally, four statistical measurements (MAPE, RMSE, MAE, and R²) were used for comparison. The results indicate that Conjugate Gradient Descent outperforms the other algorithms, yielding the highest R² and the lowest values for RMSE, MAPE, and MAE, proving its superior effectiveness in training the FFNN for forecasting monthly time series. Whereas, Batch Back-propagation was shown to be the least effective algorithm, with the lowest R² value and highest values for RMSE, MAPE, and MAE.