Stacking Optimization for Outbound Scheduling of Semi-Finished Parts Using a Double Deep Q-Network Approach

The frequent reshuffling operations and dynamic adjustments of production plans in the semi-finished part outbound process lead to low scheduling efficiency in steel structure enterprises. A novel deep reinforcement learning method, Heuristic-guided Double Deep Q-Learning (HDRL), based on mixed-integer programming and Markov Decision Process (MDP), is proposed to optimize the dynamic outbound scheduling of semi-finished parts. The method integrates multiple heuristic rules into the action space and designs state features that consider stack capacity, blocking rate, and the mixing effect of target parts, effectively improving decision accuracy and policy generalization. Experiments based on real-world data collected from a steel structure enterprise demonstrate that HDRL effectively reduces the number of reshuffling operations while maintaining millisecond-level response times.