Addressing Non-Stationarity in Reinforcement Learning via Adaptive Exploration and Distributional Robustness

Reinforcement Learning (RL) algorithms often operate under the assumption of a stationary environment. However, real-world applications frequently violate this assumption, leading to performance degradation. This paper investigates strategies for mitigating the impact of non-stationarity in RL. We propose a novel approach that combines adaptive exploration techniques with distributional robustness methods to enhance the agent's resilience to environmental changes. Our method dynamically adjusts the exploration-exploitation trade-off based on detected shifts in the state-action distribution, while simultaneously optimizing for the worst-case expected return within a defined ambiguity set. We present preliminary results demonstrating the effectiveness of our approach in a simulated non-stationary environment, showcasing improved performance compared to traditional RL algorithms.