CoMelSinger: Discrete Token-Based Zero-Shot Singing Synthesis With Structured Melody Control and Guidance

View at publisher
Authors: Junchuan Zhao, Wei Zeng, Tianle Lyu, Ye Wang
Cited by: 1
References: 51

Abstract

Singing Voice Synthesis (SVS) aims to generate expressive vocal performances from structured musical inputs such as lyrics and pitch sequences. While recent progress in discrete codec-based speech synthesis has enabled zero-shot generation via in-context learning, directly extending these techniques to SVS remains non-trivial due to the requirement for precise melody control. In particular, prompt-based generation often introduces prosody leakage, where pitch information is inadvertently entangled within the timbre prompt, compromising controllability. We present CoMelSinger, a zero-shot SVS framework that enables structured and disentangled melody control within a discrete codec modeling paradigm. Built on the non-autoregressive MaskGCT architecture, CoMelSinger re places conventional text inputs with lyric and pitch tokens, preserving in-context generalization while enhancing melody conditioning. To suppress prosody leakage, we propose a coarse to-fine contrastive learning strategy that explicitly regularizes pitch redundancy between the acoustic prompt and melody input. Furthermore, we incorporate a lightweight encoder-only Singing Voice Transcription (SVT) module to align acoustic tokens with pitch and duration, offering fine-grained frame-level supervision. Experimental results demonstrate that CoMelSinger achieves notable improvements in pitch accuracy, timbre consistency, and zero-shot transferability over competitive baselines. Audio sam ples are available at https://danny-nus.github.io/CoMelSinger/.

Fields of Study & Subjects

Speech Recognition and Synthesis Music Technology and Sound Studies Music and Audio Processing

Publication details

Journal / source
IEEE Transactions on Audio Speech and Language Processing
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Volume
34
Pages
1374–1387
Publication date
1 January 2026