Bridging domain shifts for visual perception in automated driving

Over the past decade, deep learning has been the foundation for remarkable progress and several breakthroughs in a wide range of areas like computer vision or language processing. Despite these developments, the generalization of deep neural networks to unseen domains remains a major challenge, in particular for challenging applications such as automated driving. Domain shifts, such as adverse weather conditions, can cause a significant performance degradation for the visual perception of automated vehicles, thereby limiting their ability to operate in real-world scenarios. Motivated by this, this thesis contributes to the topic of camera-based domain shift mitigation for automated driving. The focus is on unsupervised domain adaptation (UDA) and domain generalization (DG), which are the two most relevant research areas for bridging domain shifts in visual perception. Overall, this thesis makes three major contributions to these two areas. The first contribution is an extensive literature analysis for both unsupervised domain adaptation and domain generalization. It comprehensively shows the large diversity of methods, clusters them, and identifies important trends, e.g. the trend towards hybrid methods for UDA or foundation models for DG. The literature analysis serves as an important contribution to the research fields and as the foundation for the proposed methods of this thesis. The second significant contribution are input space domain generalization methods. First, this thesis studies the potential of augmentations and analyzes their interactions, but also their limitations in applicability. In the second part of this contribution, we propose a new method that achieves generalization by adaptation. For the very first time, diffusion models are used to generate a photorealistic pseudo-target domain and adapt a segmentation model to that pseudo-target domain in an unsupervised manner. The third contribution introduces a paradigm shift for UDA and DG by the efficient utilization of vision-language models. We show that significant generalization improvements can be achieved for UDA by simply equipping UDA methods with a vision-language pre-trained encoder, and for DG by simple fine-tuning a vision-language pre-trained encoder.