High-Efficiency Neural Beamforming for Real-Time Speech Enhancement on Smart Low-Power Hearable Devices

Accurate, low-latency spatial beamforming is a crucial component in emerging smart hearable devices, enabling speech enhancement while suppressing noise and interference. In this work we present an optimized methodology for real-time execution of a neural network–based minimum variance distortionless response (MVDR) beamformer on resource-constrained microcontroller units (MCUs). With mixed-precision quantization, beamforming weights can be estimated at an energy cost of 13.2 mJ, which, along a fixed-interval scheduling strategy enables end-to-end real-time operation under a 20 ms latency constraint while achieving a short-time objective intelligibility (STOI) score of 88.4. Efficiency is further improved by integrating a speech activity detection network, which bypasses speech enhancement during silence, resulting in a reduction in energy consumption to 0.62 mJ per execution with 98.5% accuracy. In realistic deployment conditions, we estimate a lifetime of 16 h on a 100 mAh battery.