Scheduling Dual-Arm Cluster Tools With Multifunctional Process Modules Using Deep Reinforcement Learning

With the advancement of semiconductor manufacturing technology, multifunctional process modules (MPMs) have been widely adopted in cluster tools to enhance production flexibility and efficiency by handling multiple operations concurrently. However, the MPMs lead to challenges in scheduling the robot due to a variety of configurations, complex robot action sequences, and regulating wafer postprocessing residency time. For scheduling such a dual-arm cluster tool (DACT) with MPMs, this article proposes a specific reinforcement learning-based scheduling method. We first develop an adaptive algorithm to generate all feasible MPM configurations. We then employ the masking technique and prioritize a replay experience buffer to improve the dueling double deep$Q$-network (D3QN), enabling it to train and identify scheduling strategies that minimize makespan and reduce wafer residency time under each configuration. We conduct experiments to demonstrate that the proposed method ensures high productivity, providing a robust and flexible scheduling solution for cluster tools in semiconductor manufacturing, and significantly enhances overall production performance.