Deep Reinforcement Learning Waypoint Generation for Attitude Station-Keeping With Sun Avoidance

The increasing saturation of low Earth orbit necessitates novel autonomous capabilities, achieved through control architectures capable of fulfilling mission objectives and operational requirements. Deep reinforcement learning offers an efficient solution by leveraging learned policies over computationally expensive optimization processes. When employed in a hierarchical architecture, learned policies can leverage existing control systems to reduce task complexity and lower computational costs. This paper examines a satellite tasked with maintaining a static orientation in Hill's frame while avoiding exposing its sensor payload to the Sun, which would render the spacecraft inoperable. This paper presents a neural network waypoint generator trained via Proximal Policy Optimization and paired with a proportional-derivative controller in a multi-rate hierarchical control architecture. Furthermore, the paper investigates the effects of observation space and action space design on the agent's performance during and after training. The performance of the control architecture is evaluated on the adherence to safety constraints, the minimization of energy used, and the time spent on task. Ultimately, agents trained with vector representations of observations and actions in the spacecraft's orbital frame exhibited the best performance.