Physical-Constraint-Embedded Deep Reinforcement Learning Approach for Flight Control of Flying-Wing Vehicle

Due to the uncertain nonlinear dynamics and the coupling motions of the longitudinal and lateral axes, the attitude control of flying-wing vehicles presents significant challenges. Proportional-Integral-Derivative (PID) control with gain scheduling relies on small disturbance linearization theory, which limits control performance under substantial nonlinear behavior. Nonlinear control techniques, such as Model Predictive Control (MPC) and Nonlinear Dynamic Inversion (NDI), are heavily dependent on accurate models, and their robustness diminishes when aerodynamic models are mismatched. To address these limitations, this paper proposes an attitude control algorithm based on Deep Reinforcement Learning (DRL), which considers the coupling of lateral and longitudinal motions as well as airspeed. The algorithm establishes a clear mapping between ’State-Action-Reward’ by utilizing an error-related state space. The proposed algorithm significantly enhances training efficiency and test success rates compared to DRL algorithms that incorporate attitude angle. Furthermore, it introduces physical constraints on the control surface deflection and an additional loss function term, resulting in smooth actuation signals and attitude responses. In simulated environments, the algorithm was compared with other state-of-the-art algorithms, and the results demonstrate its strong robustness in the presence of severe turbulence disturbances and model uncertainties. The algorithm is also assessed through real vehicle experiments, which fully corroborate its effectiveness.