AI and ML for Privacy‐Preserving Digital Twins

Reinforcement Learning (RL) is a type of machine learning, where agents learn to perform actions in an environment in order to receive the maximum cumulative reward. Earlier works have aligned the RL paradigm with deep learning and achieved state-of-the-art on large-scale sequential decision problem domains. There are some fundamental issues that still exist with the use of an approach, such as hyper-parameter sensitivity, credit assignment issue across time and long-time horizon with sparse reward, lack of diverse exploratory behaviour, and trade-off between avoiding to achieving rewards. In addition, many optimization problems in RL are hard problems, which have properties of gradient-free, non-convex, multi-modal, dynamic, and so forth, that is difficult to solve by traditional methods. In order to overcome these difficulties, the idea known as Evolutionary Reinforcement Learning has been suggested in previous research, which integrates the principles of Evolutionary Computation (EC) into the realm of Reinforcement Learning (RL). EvoRL maintains a population of agents which brings advantages like fewer amount of largely redundant info for greater robustness, more exploratory power and the capacity to score agents with episodic fitness metrics. This provides a way for agents to transfer skills learned beforehand, thus accelerating the training process and making RL applicable in scenarios that were previously impractical. This philosophy of research led to several breakthroughs in deep Reinforcement Learning, a hybrid of the advances in Deep Learning and Reinforcement Learning (RL), which has taken the space on the internet by storm, resulting in even more autonomous and adaptable systems that can now interpret incredibly high-dimensional data. Multi-Agent Reinforcement Learning (MARL) enables agents to work together or compete, providing a new way to address real-world issues that involve multiple agents1. The robustness and generalization abilities of the RL algorithms improve their resilience toward rigid and dynamic environments, paving the way for greater generalization of the RL technology. This paper functions as a reference point for both researchers and practitioners in the EvoRL sphere, pointing out notable challenges, new associated trends, and collaborative prospects for further research (for instance in optimized methodologies, official benchmarks, and scaling solutions). This facilitates the creation of more specialized benchmarks for EvoRL, aiding these researchers and practitioners to progress in the evolving hybrid area of EvoRL.