Policy Deviation Integral Guided Meta-Reinforcement Learning: Applications to High-Speed Train Trajectory Optimization

View at publisher Summarise with VIRAT
Authors: Haotong Zhang, Wanyuan Wang
Cited by: 0
References: 43

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Deep reinforcement learning (DRL) has emerged as a promising approach for the train trajectory optimization (TTO) problem in real-world high-speed rail (HSR) systems. However, there remain two issues with current DRL-based HSR operations: 1) the driver-centric markov decision process (MDP) with sparse rewards, and 2) optimizing a single rail trajectory (i.e., single task), but is less adaptable to practical HSR scenarios that require rapid responses to changing conditions (i.e., multiple tasks). In terms of sparse rewards, this paper first proposes a looped segment-wise gradient optimization (LSGO)-centric MDP that discards human-driving-pattern imitation, where the complete trajectory and terminal reward can be obtained at every agent’s action directly from the trajectory state. In terms of multiple tasks learning, meta-RL is promising to learn a policy that is capable of adapting to any new task with as little data as possible. Existing meta-RL algorithms directly use the meta parameters to train new tasks, overlooking task similarity and leading to a “lazy” agent issue with high training costs. In addressing the TTO problem, this paper finds that there exists a linear relationship between task-specific optimal policies. By fully exploiting the similarity policy between tasks, this paper proposes a policy deviation integral guided meta-reinforcement learning (PDIMRL) algorithm. It linearly adjusts and initializes new-task policies by integrating deviations between known task optima. Finally, experiments show that 1) LSGO achieves a$16.8\times $speedup in single-task training compared to driver-centric MDPs, and 2) based on LSGO-centric and driver-centric MDPs, PDIMRL only requires 41.29% and 14.74% meta-training times than benchmark meta-RL algorithms (e.g., Reptile), respectively.

Fields of Study & Subjects

Reinforcement Learning in Robotics Railway Systems and Energy Efficiency Adaptive Dynamic Programming Control

Publication details

Journal / source
IEEE Transactions on Intelligent Transportation Systems
Volume
27
Issue
8
Pages
9700–9712
Publication date
30 March 2026
Language
EN