A Review of Transformer Models and Their Variants for Predictive Tasks
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
With an increase in task complexity (e.g., long horizon, high dimensions, multivariate and multimodal),However, traditional statistical models and recurrent neural networks are limited to the issues of scalability and modelling of longrange dependencies [The Transformer].which is based on the self-attention mechanism has been proved to be an effective alternative because of its parallel computing feature and capacity to capture global temporal dependency.The purpose of this section is to provide an overview about Transformers (and predictors based on it). We first summarize the basic building blocks of the vanilla Transformer and their modifications towards time series prediction. Next,efficient attention, multi-scale temporal modelling,cross-variable dependency learning, and multimodal fusion.We summarize typical uses in the context of univariate, multivariate, and multimodal forecast settings along with comparisons between their predictive performance and computation speed. Finally,major issues including the computational complexity, robustness to distributional shift and explainability are pointed out,and possible further research directions are given.The purpose of this survey is to offer an organized point of reference for the researcher or practitioner developing Transformersbased forecasting models.
Fields of Study & Subjects
Time Series Analysis and Forecasting
Forecasting Techniques and Applications
Stock Market Forecasting Methods
Publication details
- Journal / source
- Applied and Computational Engineering
- Volume
- 230
- Issue
- 1
- Pages
- 28–37
- Publication date
- 30 March 2026
- Language
- EN