Breaking the Fog with SIGHT: Attention-Guided State Prediction for Partially Observable Reinforcement Learning

Reinforcement Learning (RL) excels in fully observable environments but faces significant challenges in partially observable scenarios, modeled as Partially Observable Markov Decision Processes (POMDPs), where agents must infer hidden states from noisy observations. We propose Sequential Inference with Guided Hidden Trajectories (SIGHT), a novel framework that integrates attention-guided hidden state prediction into an actor–critic architecture. By dynamically prioritizing relevant historical information and anticipating future dynamics, SIGHT combines the adaptability of model-free RL with the predictive foresight of model-based methods, creating a hybrid “model-based-like model-free” approach. Validated across position-based and velocity-based tasks, SIGHT outperforms state-of-the-art methods, including the POMDP Baseline, Neural Ordinary Differential Equations, and Variational Recurrent Models, achieving higher returns, stability, and adaptability. Our analysis further demonstrates the impact of sequence length, RL algorithm suitability, and attention mechanisms on performance, highlighting SIGHT’s potential for advancing RL in real-world partially observable environments.