Spiking Neural Network Gesture Recognition with Rotationally Invariant Fourier Encoding
Abstract
Gesture interfaces can free an operator from voice and touch commands when noise, vibration, or protective clothing limit traditional inputs. Yet vision models that rely on deep convolution often demand graphics hardware and watt levels far beyond the budgets of cockpit edge or cube satellite avionics. We report a complete gesture pipeline that fits in the few hundred-milliwatt envelope of an Arduino Portenta H7 paired with an OV7670 camera. A cropped hand frame is first mapped to log polar coordinates where an in-plane rotation becomes a circular shift, and a scale change becomes a straight translation. Sector averages of magnitude and wrapped phase form a compact feature vector whose elements are weighted by simple Fisher ratios so that the most discriminative frequency bands dominate the code. Each gesture class owns its own spiking neural network while sharing a single hidden layer. The network is trained using surrogate gradient backpropagation, optimizing the weights and membrane parameters to maximize a balanced accuracy score. Multiple hand gestures are recognized at forty frames per second with eighty-seven percent balanced accuracy while the board draws less than two hundred milliwatts, suggesting broader use in autonomous vehicles and wearable systems that must operate in power constrained domains.
Fields of Study & Subjects
Hand Gesture Recognition Systems
Advanced Neural Network Applications
Advanced Memory and Neural Computing
Publication details
- Journal / source
- AIAA SciTech Forum
- Publication date
- 8 January 2026