How Does Instrumental Music Help Singfake Detection?

Read full text
Authors: Xuanjun Chen, Chunhai Hu, I. N. Lin, Yi‐Cheng Lin, I-Hsiang Chiu, You Zhang, Sung-Feng Huang, Yi-Hsuan Yang, Haibin Wu, Jyh-Shing Roger Jang
Cited by: 0
References: 32

Abstract

Although many models exist to detect singing voice deepfakes (SingFake), how these models operate, particularly with instrumental accompaniment, is unclear. We investigate how instrumental music affects SingFake detection from two perspectives. To investigate the behavioral effect, we test different backbones, unpaired instrumental tracks, and frequency subbands. To analyze the representational effect, we probe how fine-tuning alters encoders’ speech and music capabilities. Our results show that instrumental accompaniment acts mainly as data augmentation rather than providing intrinsic cues (e.g., rhythm or harmony). Furthermore, fine-tuning increases reliance on shallow speaker features while reducing sensitivity to content, paralinguistic, and semantic information. These insights clarify how models exploit vocal versus instrumental cues and can inform the design of more interpretable and robust SingFake detection systems.

Fields of Study & Subjects

Music and Audio Processing Phonetics and Phonology Research Speech Recognition and Synthesis

Publication details

Journal / source
ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Publisher
IEEE
Pages
19072–19076
Publication date
21 April 2026
Language
EN