Efficient reinforcement learning for maintenance optimisation of infrastructure systems having large state and action spaces

This paper presents a methodology for optimising the maintenance of multi-component infrastructure systems, with a specific focus on bridge and road networks. The problem of Reinforcement Learning is addressed in scenarios where both the state and action spaces are combinatorial. Such settings are challenging because the policy maker’s output can be extremely large. A novel actor-critic model architecture using two neural networks is implemented to handle this complexity. It is shown that the proposed architecture results in polynomial complexity. Furthermore, this study examines the effect of balancing exploration and exploitation rates on system performance. The bridge and road infrastructure example highlights practical scenarios where it is demonstrated that, in addition to potential opportunistic and preventive maintenance behaviours in the optimal strategy, an ‘induced’ maintenance dynamic may also emerge. An in-depth analysis of the intuition and rationale behind such maintenance decisions is provided, illustrated with simple real-life example. The numerical example is presented where the goal of the decision maker is to minimise costs arising from both maintenance and user downtime. The study also includes a comparison of the computational complexities of the proposed approach and the basic Q-learning algorithm.