A Review of Transformer Models and Their Variants for Predictive Tasks

Read full text Summarise with VIRAT
Authors: Kejun Shen
Cited by: 0
References: 11

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

With an increase in task complexity (e.g., long horizon, high dimensions, multivariate and multimodal),However, traditional statistical models and recurrent neural networks are limited to the issues of scalability and modelling of longrange dependencies [The Transformer].which is based on the self-attention mechanism has been proved to be an effective alternative because of its parallel computing feature and capacity to capture global temporal dependency.The purpose of this section is to provide an overview about Transformers (and predictors based on it). We first summarize the basic building blocks of the vanilla Transformer and their modifications towards time series prediction. Next,efficient attention, multi-scale temporal modelling,cross-variable dependency learning, and multimodal fusion.We summarize typical uses in the context of univariate, multivariate, and multimodal forecast settings along with comparisons between their predictive performance and computation speed. Finally,major issues including the computational complexity, robustness to distributional shift and explainability are pointed out,and possible further research directions are given.The purpose of this survey is to offer an organized point of reference for the researcher or practitioner developing Transformersbased forecasting models.

Fields of Study & Subjects

Time Series Analysis and Forecasting Forecasting Techniques and Applications Stock Market Forecasting Methods

Publication details

Journal / source
Applied and Computational Engineering
Volume
230
Issue
1
Pages
28–37
Publication date
30 March 2026
Language
EN