Pretrained Generative Transformer (PGT) for the estimation of the response variable using causal relationships with fast convergence

The transformer architecture has been used very successfully in translating text into various languages. Additionally, transformer architecture has been successfully adapted for use in vision systems and, more recently, for estimating remaining useful life across various industrial fields. The training patterns of an artificial neural network have no relationship among the input variables. They are used to estimate response variables using a training algorithm, such as backpropagation or its variants. In the present research, an accurate estimation transformer structure is proposed to develop causal relationships between input variables and their respective training patterns and identify the importance and relevance of causal input-output relationships. The complete encoder-decoder structure is used. Input training patterns are entered into the encoder and decoder. In this way, the input data is processed by three self-attention blocks: one in the encoder and two in the decoder, allowing the model to capture dependencies between them and strengthening the base of training patterns. The system's output represents the required estimate. One of the main research contributions is to add two linear transformations at the end of the structure to identify critical and relevant input variables and training patterns. The results of a transformer structure using variable dependencies achieve fast convergence and excellent accuracy than those of a multilayer neural network trained on patterns that are unrelated to each other. Multiple simulations were carried out, and the proposed structure achieved 100% accuracy in estimates in only two epochs using the Levenberg-Marquardt algorithm, unlike the multilayer network. The architecture shown can be used across any production sector to estimate and identify the most significant variables. Smart citations: https://scite.ai/reports/10.61467/2007.1558.2026.v17i2.1271Dimensions.Open Alex.