A self-supervised multi-scale transformer for population growth and fertility intention forecasting

Accurately predicting population change trends and deeply understanding the factors influencing marriage and fertility intentions among people of childbearing age are of great significance for formulating effective population policies. However, existing population prediction models face several limitations. First, traditional Transformer architectures typically adopt a single-scale self-attention mechanism, which struggles to effectively capture the multi-scale trends inherent in population data. Second, conventional Transformer models usually rely heavily on large amounts of labeled data for supervised learning, leading to poor performance in real-world applications where labeled data are scarce or costly to obtain. To address these issues, this study proposes a Self-supervised Multi-scale Transformer model (SSMST) for population growth rate prediction, aiming to gain deeper insights into the fertility intentions of childbearing-age groups. The SSMST model effectively captures the multi-level periodic patterns in population data. Moreover, this study designs a self-supervised reconstruction task to enable effective pretraining using unlabeled data, thereby significantly reducing the model’s dependence on labeled samples. Experiments on real-world demographic datasets demonstrate that SSMST consistently outperforms strong baseline models, achieving MSE/MAE values of 0.995/0.541 and 0.762/0.516 under two experimental settings, respectively. Compared with the Transformer baseline, SSMST reduces MSE by 8.29–50.34% and MAE by 8.77–23.67%, highlighting its superior accuracy and robustness for population forecasting in low-fertility contexts.