DRL-Empowered Secure UAV-RIS Communication: A Multihead DSAC Optimization Framework

This work investigates a secure unmanned aerial vehicle (UAV)-assisted communication system enhanced by a ground-deployed reconfigurable intelligent surface (RIS), aiming to ensure secure communications for multiple ground users under the threat of a malicious eavesdropper. The core challenge lies in the joint optimization of UAV trajectory, beamforming vectors, and RIS phase shifts to maximize the average secrecy rate, which is a highly non-convex problem involving coupled continuous and discrete variables. To address this, we propose a Multi-Head Distributional Soft Actor-Critic (DSAC) framework that formulates the problem as a Markov Decision Process (MDP) and integrates continuous flight control variables including UAV position and beamforming and discrete communication variables which are RIS phase shifts into a unified reinforcement learning model. The proposed framework employs a multi-head actor network to handle mixed actions and distributional critic networks to enhance value estimation robustness. Simulation results demonstrate that the proposed Multi-Head DSAC outperforms existing reinforcement learning algorithms in terms of average secrecy rate and convergence stability, validating its effectiveness in dynamic and hostile wireless environments.