The Transformer With Extra Long Memory Language Model for Local Traditional Music

Local traditional music serves as the core carrier of regional cultural genes and bears the historical memories and emotional expressions of specific ethnic groups. Its living inheritance and digital preservation constitute a core topic in cultural heritage research. At present, local traditional music generally features complex structures, uncertain phrase durations, and variable playing techniques. Existing music analysis models based on the Transformer architecture are limited by fixed context windows and general feature embedding designs. They show obvious shortcomings in long-term sequence dependency modeling and music-specific feature capture. Such flaws make it difficult to meet the demand for high-precision digital preservation. To this end, the study proposes an improved model based on Transformer-XL to address the limitations of traditional Transformer models in processing long music sequences. This study constructs a high-precision digital preservation framework adapted to local traditional music. The model achieves performance optimization through two core modules. First, it introduces a music feature-specific embedding layer. This layer performs collaborative encoding and normalized fusion of low-level acoustic features such as pitch, rhythm, and timbre, along with high-level semantic features such as melody, genre, and instrument type. It strengthens the model’s ability to represent features in the field of traditional music. Second, it designs a dynamic attention mechanism based on spectral centroid fluctuations and rhythm density. Combined with an adaptive segmentation strategy, the mechanism realizes real-time adjustment of attention window scales according to changes in music structure. It improves the efficiency of long-term sequence dependency modeling. The study takes multi-type audio data from the Digital Platform of Chinese Traditional Music Sound Archives and the Chinese Traditional Music Resource Database Platform as experimental samples. The study also conducts comparative experiments with mainstream models around four core tasks: genre recognition, melody classification, emotion classification, and lead-accompaniment instrument recognition. It also verifies the effectiveness of core modules through ablation experiments. Experimental results show that the improved model exhibits remarkable advantages in all tasks. The accuracy of genre recognition increases by at least 12.7% compared with contrast models. The advantage in emotion classification metrics is no less than 10.5%. The accuracy of lead-accompaniment instrument recognition rises by more than 10.9%. The accuracy of melody classification improves by at least 10.1%. Ablation experiments confirm that the synergy of the two core modules is the key to performance breakthroughs. The model’s improvement rate drops sharply to about 3% after removing either module. A single module can only achieve a performance improvement of 7%–9%. This study deeply integrates the Transformer-XL model with local traditional music research and constructs a technical system of “feature fusion-dynamic modeling-full-chain preservation”. It breaks through the technical bottleneck of long music sequence analysis and realizes high-precision and multi-dimensional collection and analysis of traditional music data. The study provides a new technical path for the digital preservation of local traditional music. At the same time, it offers a reference methodology for research in the interdisciplinary field of artificial intelligence and traditional music.