Ultrafast visual perception beyond human capabilities enabled by motion analysis using synaptic transistors

Optical flow, inspired by biological visual systems, calculates spatial motion vectors aiming to enable robotics to excel in dynamic environments. However, current algorithms, despite human-competitive task performance on benchmark datasets, suffer from significant time delays, limiting practical deployment. Here, we introduce a neuromorphic temporal-attention hardware that emulates the interaction between the retina and the lateral geniculate nucleus (LGN) to extract temporal motion cues directly in hardware. Using a two-dimensional synaptic transistor array, the system encodes brightness changes and accumulates them in analog, non-volatile states, generating compact regions of interest (ROIs). These ROIs then act as inputs to conventional downstream optical flow and vision algorithms, enabling ultrafast motion analysis. At the hardware level, the synaptic transistor offers high-frequency response (~100 μs), non-volatility (>10,000 s), and endurance (>8,000 cycles). Compared to state-of-the-art algorithms, our approach demonstrates a 400% speedup, surpassing human-level performance while maintaining or improving accuracy through temporal priors. Human visual system relies on temporal attention to detect moving objects before high-level processing with large computational overheads. Wang et al. emulate this function in a neuromorphic hardware, showing a 400% speedup compared to algorithm-based visual perception and surpassing human capabilities.