Het opleggen van beperkingen aan modelvrije versterkende leeralgoritmes met als toepassing autonoom rijden

Designing a fully autonomous vehicle that can operate flawlessly under all conditions is a very challenging task for which no all-encompassing solution has been found yet. For now, the problem is typically divided into separate subproblems and tasks for which extra assumptions can be made, or for which certain conditions can be relaxed. In this doctoral thesis, we specifically focus on the high-level decision making process, in charge of selecting suitable reference signals (tracked by lower level modules) to efficiently and safely navigate traffic on highways. Various strategies have already been developed to tackle this complex decision making task, ranging from manually designed rule-based controllers to state-of-the-art machine learning models. Manually designing control laws quickly becomes infeasible, as it requires one to anticipate on all possible traffic situations beforehand. In contrast, using reinforcement learning we can let the vehicle explore and continuously learn on its own, making it quite an appealing approach in this context. Although standard model-free reinforcement learning methods can automatically learn suitable driving policies through trial-and-error interactions with the environment, they lack an inherent mechanism to impose hard (safety) constraints on the trained policies. The most straightforward way to deal with constraints, is to add extra penalties to the reward signal whenever a constraint is violated. However, this approach can be very time consuming and error-prone, as it requires to manually handcraft penalties and to perform large scale experiments to find the right balance between the different reward components. Therefore, in this dissertation, we develop reinforcement learning methods with embedded support for various types of (hard) constraints, and investigate various approaches for integrating prior knowledge about the problem directly into the model-free reinforcement learning process. Our contributions offer the designer a comprehensive toolbox of policy constraints that can be seamlessly combined, and that are automatically taken care of during training. This simplifies the design process and leads to a more natural problem formulation, allowing the designer to clearly separate objectives from constraints. In particular, we have made following contributions: (i) enforcing hard state-dependent bounds on the policy's actions, (ii) obtaining local stability guarantees by constraining the policy's weight space, (iii) reducing temporal roughness of policy actions through smoothness regularization, (iv) developing a theoretical primal-dual framework for analyzing various policy constraints, such as transition or action density constraints, which provides novel insights into the intrinsic relationship between policy constraints and learnable reward modifications, and (v) introducing practical hierarchical control setups using options with embedded safety and comfort constraints. For each of these contributions, we derive practical algorithms following state-of-the-art reinforcement learning techniques. The proposed methods are evaluated in realistic scenarios, such as the autonomous driving applications on highways, to demonstrate their effectiveness. When correctly applied, the enforced constraints can provide strong safety guarantees, as well as improve the performance and convergence speed. Furthermore, they can result in more interpretable, trustworthy and comfortable driving policies that have the same expressiveness and flexibility as human drivers. In this regard, of all the investigated approaches, the policies using hybrid options perform the best under varying traffic conditions.