Fractional-Order Soft Actor--Critic for Vision-Based UAV Navigation With Structured Experience Replay

Deep reinforcement learning holds significant potential for continuous control tasks, but its real-world application is often limited by algorithmic instability and poor sample efficiency. Although Soft Actor-Critic (SAC) is a strong and widely adopted baseline for continuous control, its standard temperature adaptation and uniform replay mechanism may limit training stability and data utilization in complex vision-based navigation tasks. To overcome these limitations, we introduce the Fractional-order Temperature Adaptive Soft Actor-Critic (FSAC) framework. FSAC integrates two core innovations: a fractional-order temperature controller that uses Grünwald-Letnikov derivatives to incorporate historical gradient information, thereby stabilizing entropy tuning and damping oscillatory dynamics; and a quadrant-based experience replay (QER) strategy that jointly rank-normalizes transition Q-values and temporal-difference errors to construct balanced mini-batches, improving sample informativeness and distributional diversity without relying on hyperparameter-sensitive prioritization. Experiments on OpenAI Gym benchmarks show that FSAC improves stability and performance in most settings. Furthermore, when deployed with a curriculum learning strategy in high-fidelity AirSim simulations for unmanned aerial vehicle (UAV) navigation across simple, dense, and dynamic obstacle environments, FSAC exhibits improved training stability and generalization performance, particularly in dynamic scenarios. These results indicate that FSAC provides a robust and efficient extension of SAC for complex, non-stationary control problems.