Knowledge-guided reinforcement learning for preventive maintenance planning in economically dependent multi-component systems

Many engineering systems comprise multiple components that operate simultaneously. These components degrade gradually and irreversibly at varying rates, eventually leading to failure. Preventive maintenance is crucial for reducing the likelihood of failures and associated costs. In many systems, inspections are costly, requiring the joint optimization of inspection and maintenance policies. In multi-component settings, maintenance actions are often economically interdependent due to operational shutdowns and shared labor, resulting in high setup costs. We aim to identify an optimal preventive maintenance and inspection policy that minimizes total costs in economically dependent multi-component systems, where each component degrades stochastically at random rates. Planning with continuous degradation measurements is challenging because the action space grows exponentially with the number of components, and the continuous state space is vast. Furthermore, economic dependencies introduce another layer of complexity. Existing decision-making methods, including deep reinforcement learning, often struggle with these challenges. We propose a knowledge-guided reinforcement learning approach that incorporates: (1) a sparsity-enhanced stochastic hierarchical policy, (2) a monotonic maintenance policy, and (3) degradation estimation with reward shaping based on Bayesian predictions. This framework effectively captures the infrequent nature of maintenance events. Case and numerical studies demonstrate substantial cost reductions and performance improvements over existing methods.