EffiFusion-GAN: Efficient Fusion Generative Adversarial Network for Speech Enhancement

Abstract Speech enhancement models increasingly estimate magnitude and phase jointly, but the resulting architectures can be difficult to deploy under tight model-size constraints. We present EffiFusion-GAN, a time–frequency generative adversarial network designed to preserve enhancement quality with fewer trainable parameters. The generator combines a depthwise-separable dense encoder, residual time–frequency Conformer blocks, and parallel magnitude-mask and phase decoders; L1 unstructured pruning removes 30% of the selected convolutional weights. On VoiceBank+DEMAND, EffiFusion-GAN achieves PESQ 3.45, CSIG 4.71, CBAK 3.91, COVL 4.18, and STOI 0.96 with a reported parameter count of 1.08 million. Its PESQ is 0.05 below MP-SENet while its reported parameter count is 47% lower. Ablation results show that replacing depthwise-separable convolution with standard convolution nearly doubles the parameter count for a 0.02 PESQ increase, whereas removing residual connections reduces PESQ by 0.10. These findings support a favorable parameter–quality trade-off. Future work should confirm latency, memory use, and robustness on deployment hardware and unseen acoustic conditions.