Combining Supervised Pretraining and Reinforcement Learning for Scalable Low-Thrust Spacecraft Trajectory Design

This paper investigates a hybrid training framework that combines supervised pretraining with deep reinforcement learning to approximate optimal low–thrust control for interplanetary transfers. The primary test case is a circular Two–Body Transfer problem in which a continuously thrusting spacecraft is transferred between heliocentric orbits of varying semi–major axis while minimizing propellant consumption. Mass–optimal reference trajectories are first generated using an indirect optimal control formulation with Pontryagin’s Maximum Principle and homotopic smoothing to obtain bang–bang thrust profiles. These trajectories are then converted into a Markov Decision Process dataset by mapping each state and optimal control into a normalized polar state representation and continuous action vector, and encoded as state–action–reward–transition tuples that populate the replay buffer of a Soft Actor–Critic (SAC) agent. Comparative experiments against a baseline SAC agent trained from scratch shows improvements in average episode reward and higher performing controllers when using pretraining in Monte Carlo validation tests. These results demonstrate that seeding off–policy reinforcement learning with mass–optimal trajectory data is an effective strategy for improving training efficiency in reinforcement learning applied to trajectory design problems.