DiTSinger: Scaling Singing Voice Synthesis with Diffusion Transformer and Implicit Alignment

Read full text
Authors: Zepeng Du, Guilin Deng, Xiaofeng Guo, Xin Gao, Linke Li, K.W.E. Cheng, Fubo Han, Siyu Yang, Peng Liu, Pan Zhong, Qiang Fu
Cited by: 0
References: 20

Abstract

Recent progress in diffusion-based Singing Voice Synthesis (SVS) demonstrates strong expressiveness but remains limited by data scarcity and model scalability. We introduce a two-stage pipeline: a compact seed set of human-sung recordings is constructed by pairing fixed melodies with diverse LLM-generated lyrics, and melody-specific models are trained to synthesize over 500 hours of high-quality Chinese singing data. Building on this corpus, we propose DiTSinger, a Diffusion Transformer with RoPE and qk-norm, systematically scaled in depth, width, and resolution for enhanced fidelity. Furthermore, we design an implicit alignment mechanism that obviates phoneme-level duration labels by constraining phoneme-to-acoustic attention within character-level spans, thereby improving robustness under noisy or uncertain alignments. Extensive experiments validate that our approach enables scalable, alignment-free, and high-fidelity SVS.

Fields of Study & Subjects

Speech Recognition and Synthesis Music Technology and Sound Studies Generative Adversarial Networks and Image Synthesis

Publication details

Journal / source
ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Publisher
IEEE
Pages
17717–17721
Publication date
21 April 2026
Language
EN