LSTM-augmented DQN for quantitative trading in partially observable markets

Deep reinforcement learning (DRL) has demonstrated tremendous potential in quantitative trading research in recent years. However, its performance still faces significant challenges in financial market environments characterized by high noise, low signal-to-noise ratios, and partial observability. Traditional approaches often rely on single supervised learning or reinforcement learning architectures. Still, these methods frequently struggle to adequately capture temporal dependencies and may exhibit convergence instability or overfitting in complex trading scenarios. To address these challenges, this paper proposes an LSTM–DQN-based quantitative trading framework. This framework first employs LSTM to extract latent dynamic features from stock price sequences, then constructs a discrete state space using dual moving averages to achieve efficient encoding of market trends. Subsequently, it performs policy optimization via DQN within the built environment, effectively mitigating partial observability issues while achieving breakthroughs in learning efficiency and policy stability. This paper comprehensively considers returns, transaction costs, position constraints, and risk exposure in reward function design, ensuring the strategy’s robustness in high-volatility environments. We conducted experiments using three representative stocks from China’s A-share market and performed ablation analysis to validate the contribution of key features to model performance. Empirical results indicate that our proposed method outperforms the comparative strategy in terms of cumulative return, annualized return, and Sharpe ratio, with the cumulative return and annualized return increasing by a maximum of 16.41% and 14.87%, respectively.