Automatic generation of labanotation based on a hybrid transformer–LSTM network with multi-scale spatio-temporal features

To support the digital preservation and dissemination of folk dance as an intangible cultural heritage, the automatic generation of Labanotation from motion capture data has become a prominent research direction. However, existing techniques show limited capability in extracting skeletal features and fail to adequately capture the spatio-temporal dependencies inherent in motion sequences. To address these limitations, we propose the Multi-Scale Spatio-Temporal Hybrid Transformer-LSTM (MST-HTL) model, which aims to improve both the accuracy and robustness of automatic Labanotation generation. First, a multi-scale spatio-temporal convolutional network is developed to extract fine-grained local skeletal motion features, thereby enhancing the representation of complex dance movements. Next, the hybrid architecture integrates a Transformer encoder with a dynamically gated residual structure and an LSTM decoder enhanced by an enhanced attention mechanism. Together, these components jointly global spatio-temporal representations while reinforcing local details, leading to a more comprehensive modeling of motion dependencies. Experiments on two benchmark Laban datasets show that MST-HTL outperforms prior methods, achieving a 0.92% improvement on LabanSeq16 and a 1.43% improvement on LabanSeq48. This work provides a solid theoretical and methodological foundation for the digital preservation of dance-related intangible cultural heritage.