Spatial post-filtering for speech enhancement and source separation

Microphone arrays with limited spatial resolution, such as compact and low-order configurations, present challenges for spatial audio processing tasks including sound source separation and interference suppression. Traditional beamforming methods alone may lack the robustness and adaptability required for practical audio scenarios, particularly under constrained sensor arrangements. Further, despite advancements in spatial audio, most common spatial recording devices only utilize low-order, often first-order, directional signals. This thesis addresses these challenges by developing spatial post-filtering techniques inspired by the Cross-Pattern Coherence (CroPaC) algorithm, aiming to enhance the performance of omnidirectional and low-order directional microphone arrays. First, we formulate a CroPaC-based post-filter estimated from low-order beamformer outputs. Even when the array departs from ideal geometric assumptions, simulations in multi-speaker conditions demonstrate consistent suppression of interference and background noise. Second, we investigate a space-domain variant (SD-CroPaC) for linear arrays with a focus on speech enhancement. Using single- and dual-line uniform linear arrays (ULAs), common beamforming ambiguities (e.g., front–back) can be suppressed. A combination that uses two post-filters yields stronger separation than either filter alone while preserving target speech. Third, building on these insights, we propose a non-linear combination strategy that optimizes the merging of multiple CroPaC post-filters estimated from low-order directional signals. This approach improves spatial selectivity and hence interferer suppression, and outperforms a higher-order CroPaC baseline under the same conditions. Finally, we generalize coherence-driven post-filtering to distributed omnidirectional microphones by introducing aggregated pairwise similarity measures. This results in a soft mask that remains effective when sensors are spatially separated, with positional errors, and in a reverberant acoustic environment. Collectively, these contributions significantly advance spatial audio processing techniques, providing robust and practical solutions to improve spatial selectivity and interference suppression capabilities of low-order and distributed microphone array systems, with applications to portable recorders, headsets, smartphones, and distributed omnidirectional microphone systems.