Reinforcement learning-based guidance law for highly-maneuvering targets

In the terminal guidance stage, when the target's maneuverability approaches that of the interceptor, traditional guidance laws are prone to overloading saturation, resulting in a decrease in interception accuracy. To address this problem of intercepting highly-maneuvering targets, a softmax-based reinforcement learning (SMRL) guidance law is proposed. To overcome the susceptibility of conventional reinforcement learning-based guidance laws to value estimation bias, this guidance law combines the softmax deep double deterministic policy gradients (SD3) algorithm, which significantly reduces inaccuracies in optimal policy approximation due to value estimation bias. Furthermore, the interception problem is formulated as a Markov decision process (MDP), and a dimensionless state normalization method is introduced to enhance the generalization ability of guidance law. A heuristic reward function tailored for highly-maneuvering targets is developed, considering the characteristics of aircraft autopilots and actuator saturation. Simulation results demonstrate that SMRL outperforms conventional methods in terms of generalization capability, guidance accuracy, and maximum tolerable overload ratio, exhibiting great potential for practical engineering applications in high-maneuverability interception missions.