Stability-Guaranteed Grant-Free Access for Cyber–Physical System over Space–Air–Ground Integrated Networks

In this paper, we investigate the grant-free (GF) accessing for cyber–physical systems (CPSs) over space–air–ground integrated networks (SAGINs) by jointly considering system stability and power consumption. The problem of GF access for CPSs over SAGINs is modeled as a Markov decision process where preamble sequences are chosen to minimize power consumption while guaranteeing system stability. To solve this problem, a distributed multi-agent deep reinforcement learning framework based on factorization technology is proposed. In addition, a local network based on hierarchical reinforcement learning is designed to prevent the explosion of the dimension of the action space, in turn reducing the computational complexity of the proposed algorithm. Finally, the simulation results validate the performance superiority of the proposed scheme in terms of convergence, power consumption and stability compared with the baseline schemes.