Joint Sustainable Control and Quantum Reinforcement Learning for Energy-Efficient Cube-Satellite Networks

Satellites have been envisioned as primary non-terrestrial networks capable of seamless global network and surveillance services. Among various satellite types, Cube Satellites (CubeSats) have been actively researched because multiple CubeSats can be conveniently positioned in a target orbit simultaneously and in proximity to Earth. However, CubeSats are small-scale, and thus, they are not able to accommodate a sizable battery, imposing constraints on the duration of their mission. Considering this energy limitation, in order to realize global network services using multiple CubeSats, this paper proposes a novel two-stage Reinforcement Learning (RL) algorithm for energy-efficient CubeSats where RL is utilized for dynamic control under uncertainty. Firstly, sustainable control for single-CubeSat orbital maneuver is considered using deep deterministic policy gradient for vertical position adjustment over a continuous action domain. Secondly, a novel quantum multi-agent RL algorithm for multi-CubeSat cooperative scheduling is designed to realize action dimension reduction into a logarithmic scale based on our proposed Projection-Valued Measure (PVM) over the quantum domain. It is highlighted that our considering two single- and multi-CubeSat problems cannot be separately considered for extreme energy management. The performance evaluation results demonstrate that the proposed algorithm outperforms other benchmarks with 1.51× higher performance in orbital control, 2.71× higher converged reward in enormous action dimensions, and 2.27× higher average network performance.