One Timestep Spiking Actor Network with Adaptive Global-Connected Encoding and Threshold Learning

Spiking neural networks (SNNs) have garnered significant attention in reinforcement learning tasks for their low power consumption. However, traditional spiking reinforcement learning (SRL) methods, which rely on local-connected encoding and fixed-threshold learning, struggle to capture the inter-dimensional correlations of input information within short timesteps, limiting the network’s expressive capacity. While increasing timesteps can significantly enhance performance, excessive timesteps result in substantial delays. In this paper, we propose an adaptive global-connected encoding method that directly captures global dependencies (such as long-range feature interactions), allowing for efficient encoding of input state information even within a single timestep. Additionally, threshold-learnable spiking neurons are introduced to effectively capture and distinguish subtle changes in input signals at each timestep, thereby enhancing the spatial-temporal state representation during spike information transmission. Extensive experimental results demonstrate that the proposed method achieves performance comparable to state-of-the-art algorithms using only a single timestep, while significantly reducing inference latency and energy consumption. When extended to multiple timesteps, our approach consistently outperforms existing methods, showing substantial improvements across eight continuous control tasks from OpenAI Gym.