Hardware-software collaborative computing of photonic spiking reinforcement learning for robotic continuous control

Robotic continuous control tasks impose stringent demands on the energy efficiency and latency of computing architectures due to their high-dimensional state spaces and real-time interaction requirements. Conventional electronic computing platforms face computational bottlenecks, whereas the fusion of photonic computing and spiking reinforcement learning (RL) offers a promising alternative. We propose a computing architecture based on photonic spiking RL, which integrates the twin delayed deep deterministic policy gradient algorithm with a spiking neural network. The proposed architecture employs an optical-electronic hybrid computing paradigm wherein a silicon photonic Mach-Zehnder interferometer (MZI) chip executes linear matrix computations, while nonlinear spiking activations are performed in the electronic domain. Experimental validation on the Pendulum-v1 and HalfCheetah-v2 benchmarks demonstrates the capability of the system for software-hardware coinference, achieving a control policy reward of 5497±251 on HalfCheetah-v2, a 15% reduction in convergence steps, and an action deviation below 2.2%. Notably, we present the first application of a programmable MZI photonic computing chip to robotic continuous control tasks, attaining an energy efficiency of 1.39 TOPS/W and an ultralow computational latency of 120 ps. Such performance underscores the promise of photonic spiking RL for real-time decision-making in autonomous and industrial robotic systems.