CuFusion-MCTS: A Persistent-RNG Framework for GPU-Native Monte Carlo Planning

Monte Carlo Tree Search (MCTS) is an algorithm that is employed to solve decision-making problems in complex systems but it is numerically expensive when the discrete action space is large. In this paper, we provide an MCTS selection engine which takes the advantage of graphical processing units. The engine is done through fused CUDA kernels, where the action scoring, UCB1 evaluation and Gumbel-based exploration are combined into one single executable. Iteration offers cross-border persistent cuRAND states and hierarchical reduction scheme offers good scalability with regard to action space dimensionality. On CartPole, Taxi, Chess, and Go CartPole Benchmarking on CartPole, Taxi, Chess, and Go environments has shown a steady performance improvement of between 50 and 100 times compared to CPU implementations with 1,200 actions per second using under 80MB of memory. The suggested framework enables high-scale policy optimization on-the-fly and provides a full-fledged reinforcement-learning and planning workflow, thus providing meaningful enhancements to the future decision-making systems that are based on GPUs.