Deep Reinforcement Learning for Near-Optimal Control Sequence in Discrete Event Systems

This work investigates the use of deep reinforcement learning to address the scheduling problem of identifying minimal control sequences while ensuring deadlock avoidance in transition-timed Petri nets. Traditional control strategies often depend on exhaustive search algorithms or heuristics, both of which tend to be computationally intensive and exhibit poor scalability as system complexity increases. In this study, we formulate the control sequence scheduling task as a Markov decision process and adopt the Deep Q-Network framework to learn control policies through interaction with an integrated timed Petri net simulation environment. The reward function is specifically designed to minimize total execution time, with penalties applied to extended durations, thereby guiding the learning process toward efficient and safe behaviors. In addition, we evaluate the robustness of the learned policy under structural perturbations (i.e., failed transitions) and temporal variations via scaled firing delays. Experimental results indicate that the proposed method consistently identifies control sequences with shorter makespans when compared to conventional approaches, offering significant improvements in both runtime and computational efficiency. Moreover, once the learned policy generalizes effectively, it can be reused to quickly generate feasible control sequences even under moderate perturbations, without requiring retraining. The comparative analysis further highlights these advantages, demonstrating the potential of the proposed reinforcement learning-based framework as a practical and scalable solution for optimizing control strategies in complex timed Petri net models. Two case studies on manufacturing systems are used to illustrated the efficiency of the proposed strategy.