Distinguishability-Driven Voice Generation for Speaker Anonymization via Random Projection and GMM

Speaker anonymization effectively conceals speaker identity in speech signals to protect privacy. To address issues in existing anonymization systems, including reduced voice distinguishability, limited anonymized voices, reliance on an external speaker pool, and vulnerability to privacy leakage against strong attackers, a novel distinguishability-driven voice generation for speaker anonymization via random projection and the Gaussian Mixture Model (GMM) is proposed. This method first applies the random projection to lower the dimensionality of the X-vectors from an external speaker pool, and then constructs a GMM in the reduced dimensional space to fit the generative model. By sampling from this generative model, anonymous speaker identity representations are generated, ultimately synthesizing anonymized speech that maintains both intelligibility and distinguishability. To ensure the anonymized speech remains sufficiently distinguishable from the original and prevents excessive similarity, a cosine similarity check is implemented between the original X-vector and pseudo-X-vector. Experimental results on the VoicePrivacy Challenge datasets demonstrate that the proposed method not only effectively protects speaker privacy across different attack scenarios but also preserves speech content integrity while significantly enhancing speaker distinguishability between original speakers and their corresponding pseudo-speakers, as well as among different pseudo-speakers.