Modern methods of Document processing for Calculating Stock Market indicators

This article discusses modern methods of extrapolating pre-trained transformers aimed at improving their ability to process long and short text sequences in Russian in the financial sector. Particular attention is paid to the task of classifying texts that reflect broker analysts’ expectations regarding market movements (expectations of growth, decline, or uncertainty of change). To solve this problem, the application of lightweight language models ruBERTtiny1 and ruBERT-tiny2 is investigated, which are adapted to work effectively with large amounts of input data while maintaining prediction quality. The paper analyzes various approaches to expanding the contextual window of models, including extrapolation methods, and considers the impact of tokenization, vectorization, and embedding strategies on the final classification results. Additionally, the paper discusses the peculiarities of using transformers in conditions of increased market volatility and changing news flows, which allows for a more in-depth assessment of the stability of the proposed solutions. Furthermore, a formula for calculating a leading indicator for stock markets is proposed and discussed, demonstrating the practical significance of using transformer models in the analysis of financial texts and the formation of analytical metrics. The presented results highlight the promising application of compact transformers in predictive financial analytics tasks.