Sparse transmission in diffractive deep neural networks via optical spiking neurons

Optical neural networks (ONNs) perform extensive computations using photons instead of electrons, resulting in passively energy-efficient and low-latency computing. Among various ONNs, diffractive optical neural networks (DONNs) particularly excel in energy efficiency, bandwidth, and parallelism, therefore attracting considerable attention. However, their performance is limited by the inherent constraints of traditional frame-based sensors, which process and produce dense and redundant information at low operating frequency. Inspired by the spiking neurons in the human neural system, which utilize a thresholding mechanism to transmit information sparsely and efficiently, we propose integrating a threshold-locking method into neuromorphic vision sensors to generate sparse and binary information, achieving microsecond-level accurate perception similar to human spiking neurons. By introducing binary dual adaptive training (BAT), the high-speed, spike-based diffractive optical neural network (S 2 NN) demonstrates an ultra-fast single-layer inference operating speed of 2036 FPS and accurate performance, which is 20-fold faster than the previous reported results from the DONN with similar platform but frame-based processing. Our work demonstrates the potential of incorporating neuromorphic architecture to facilitate optical neural network applications in real-world scenarios machine vision tasks.