Joint Optimization of Maintenance, Inventory, and Remanufacturing for Multiunit Systems: A Deep Reinforcement Learning Approach

The joint optimization of maintenance and spare parts inventory enables overall cost-effective decisions by integrating both system condition monitoring data and real-time inventory information. As sustainable manufacturing continues to evolve, remanufacturing is emerging as an essential strategy for extending component lifecycles and reducing environmental and economic costs. Nevertheless, recent studies have rarely integrated component remanufacturing into a dynamic decision framework for condition-based maintenance (CBM) and spare parts provisioning driven by system states, which will result in increased computational complexity. This paper investigates the joint optimization of CBM, spare parts inventory, and component remanufacturing for a multi unit system with degradation randomness. Firstly, we formulate this joint decision-making problem as a Markov Decision Process (MDP) that incorporates system states with degradation randomness and spare parts inventory accounting for multi-lifecycle remanufacturing. Building upon widely adopted threshold control methodologies, we design a heuristic Threshold-Based Policy to derive simple solutions to joint maintenance-inventory control with remanufacturing. However, the computational complexity induced by degradation randomness and remanufacturing severely limits the solution efficiency of the Threshold Based Policy. To further reduce computational time, we propose a Threshold-Optimized Deep Reinforcement Learning (TDRL) algorithm that integrates a threshold initialized policy, maintenance action filtering, and state standardization. A case study on a ball screw system demonstrates that the proposed TDRL algorithm outperforms the Threshold-Based Policy in terms of both cost and computational time. Sensitivity analyses are conducted to analyze the impact of key factors on the benefits of the remanufacturing policy within the joint optimization framework.