Advancing AlphaZero: Enhanced Search, Modern Architectures, and Games with Hidden Information

AlphaZero was a groundbreaking advancement in the field of artificial intelligence, especially for games with perfect information and a large state and action space. Building on the successes in classic board games such as Go, Chess and Shogi, MuZero was a first step towards games with hidden information. However, many challenges remain, especially in terms of transferability to more complex environments. In this thesis, the basics of the AlphaZero framework are first introduced using the Chess variant Crazyhouse, a domain with extended action space through the re-insertion of captured pieces. The technical components of AlphaZero are discussed, including the neural network and the Monte-Carlo Tree Search (MCTS). In the main part of the thesis, AlphaZero is further refined in several ways. First, the search algorithm is improved by introducing the Monte-Carlo Graph Search (MCGS), which explicitly considers transpositions and thus enables a more efficient reuse of partial searches. Secondly, the input representation is revised and modern network architectures such as transformer models are used to increase performance. Thirdly, a modular search approach with a Mixture-of-Experts model is presented, which uses different experts specifically in different game phases and thus allows strategic specialization. In addition, AlphaZero is extended to more complex scenarios, including multi-agent environments such as Pommerman with more than two players and games with incomplete information. These extensions demonstrate the versatility of the framework and provide initial approaches to modeling uncertainty and opponent behavior. Finally, the thesis discusses the limitations of existing approaches and possible perspectives for future developments. The contributions presented thus take an important step towards modular, adaptable AI systems that can be used across domains.