mmWave Radar Aware Dual-Conditioned GAN for Speech Reconstruction of Signals With Low SNR

Millimeter-wave (mmWave) radar captures are band-limited and noisy, making for difficult reconstruction of intelligible full-bandwidth speech.In this work, we propose a twostage speech reconstruction pipeline for mmWave using a Radar-Aware Dual-conditioned Generative Adversarial Network (RAD-GAN), which is capable of performing bandwidth extension on signals with low signal-to-noise ratios (-5 dB to -1 dB), captured through glass walls.We propose an mmWavetailored Multi-Mel Discriminator (MMD) and a Residual Fusion Gate (RFG) to enhance the generator input to process multiple conditioning channels.The proposed two-stage pipeline involves pre-training the model on synthetically clipped clean speech and fine-tuning on fused mel spectrograms generated by the RFG.We empirically show that the proposed method, trained on a limited dataset, with no pre-trained modules, and no data augmentations, outperformed state-of-the-art approaches for this specific task.