Efficient and Secure UAV-Assisted Industrial Internet of Things Based on Confidence-Weighted Reinforcement Learning

Both the security and the freshness of data are crucial in the Industrial Internet of Things (IIoT). Unmanned aerial vehicles (UAVs) have been widely used for sensing data collection in IIoT due to their high flexibility, low cost, and easy accessibility. However, the issue of trajectory planning in multi-UAV-assisted IIoT has not been adequately studied with regard to the security threat mitigation and the freshness of data. Therefore, we investigate the trajectory planning problem for secure data collection in a multi-UAV-assisted IIoT network, using UAVs to collect data timely while detecting and mitigating the effects of ground malicious users. In the envisioned network, some UAVs act as airborne base stations to collect data, and the others act as jammers to emit artificial noises to interfere with the monitoring. In order to minimize the average information age (AoI) of collected data, we propose a novel confidence-weighted twin delayed deep deterministic policy gradient (CTD3) algorithm for multiple UAV trajectory optimization. In this algorithm, UAVs are modeled as an agent (i.e., a cognitive entity) that is capable of planning their flight trajectory based on the target of minimizing the AoI of collected data and the perceived information of the surrounding environment. In addition, we innovatively introduced random network distillation (RND) technology into reinforcement learning and used its weighted the loss function of policy network to guide the UAV to better utilize the empirical data from its interactions with the environment to continuously learn and optimize its behavioral policies, thereby improving the efficiency of data utilization. Our simulation results demonstrate the effectiveness and superiority of the CTD3 algorithm in terms of both the cumulative discount reward and the average AoI.