Edit Content, Preserve Acoustics: Imperceptible Text-Based Speech Editing via Self-Consistency Rewards

Read full text
Authors: Yong Ren, Jiangyan Yi, Jianhua Tao, Wang, Tao, Le Xu, Zhengqi Wen
Cited by: 0

Abstract

Imperceptible text-based speech editing modifies spoken content through transcript manipulation while preserving acoustic continuity. Prior acoustic-space approaches suffer from content-style entanglement, causing unstable generation and boundary artifacts. We introduce a framework guided by the principle of "Edit Content, Preserve Acoustics". Editing is conducted in a stable semantic space, while acoustic realization is handled by a Flow Matching decoder. To ensure perceptual consistency, we propose Self-Consistency Rewards Group Relative Policy Optimization, which leverages a pre-trained Text-to-Speech model as an implicit critic, together with intelligibility and duration constraints. Experiments demonstrate consistent improvements over state-of-the-art autoregressive and non-autoregressive baselines in intelligibility, robustness, and perceptual quality.

Fields of Study & Subjects

Speech Recognition and Synthesis Speech and Audio Processing Generative Adversarial Networks and Image Synthesis

Publication details

Journal / source
arXiv (Cornell University)
Publication date
31 January 2026
Document type
Preprint