Luottamusalueet ja data-augmentointi esitysoppimisen latenttiavaruudessa vahvistusoppimisessa - Menetelmä esitysten johdonmukaisuuden ja oppimisen vakauden parantamiseksi

Summarise with VIRAT
Authors: Karri Majoinen
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Reinforcement learning (RL) has seen great successes in recent years across domains. Despite these successes, sample efficiency has remained a relevant problem in RL, especially when attempting to learn policies from high-dimensional inputs, such as images. Representation learning is one way of addressing this challenge using, e.g., technologies like technologies like autoencoders to learn a lower dimensional latent space from which the agent can learn its policy. There are varied approaches to learning the latent space, with methods like world models, contrastive learning, reconstruction-based autoencoders, and encoders based on bisimulation metrics. While these methods alleviate the issue with sample efficiency, they introduce novel problems with training stability due to inconsistent representation. In addition to representation learning, data augmentation like DrQ have shown state-of-the-art results in improving sample efficiency. The idea behind such methods is to reduce variance in Q-value estimation by augmenting the observations of the agent. To address the stability issues of current representation-based methods, and inspired by their success in other RL algorithms, this thesis introduces the methodology of trust region (TR) optimization for representation learning. Furthermore, inspired by methods of data augmentation in image spaces, we present a method for augmenting the latent space of a deep autoencoder to decrease variance in Q-value estimation increase sample efficiency. We evaluate the efficacy of our methodologies against existing representation learning approaches. Our experiments validate that trust region optimization improves representation consistency, and show that it increases stability and sample efficiency across encoder architectures. On the contrary, we show observed strongly negative results if latent data augmentation with respect to baseline methods. This thesis provides a foundation an architecture-agnostic optimization method to increase training stability of deep encoders.

Fields of Study & Subjects

Reinforcement Learning in Robotics Domain Adaptation and Few-Shot Learning Machine Learning and Algorithms

Publication details

Journal / source
Aaltodoc (Aalto University)
Publication date
31 January 2026
Language
EN
Document type
Thesis