Autoencoders

This chapter explores autoencoders, a type of neural network designed to replicate their input to the output, focusing on their structure, applications and relationship with classical dimension reduction techniques such as principal component analysis (PCA) and singular value decomposition (SVD). Autoencoders typically approximate rather than perfectly replicate their input, often through undercomplete hidden layers or regularisation methods. The chapter discusses various types of autoencoders, including sparse and denoising autoencoders, and their applications in dimension reduction and noise removal. It also examines the relationship between autoencoders and generative models, with a specific focus on variational autoencoders. The chapter highlights that autoencoders are a subset of deep neural networks and thus benefit from the theoretical and practical developments in this field. It provides a detailed comparison between autoencoders and classical dimension reduction techniques, such as PCA and SVD, and discusses their respective advantages and applications. The chapter compares dimension reduction and data reconstruction techniques using PCA, SVD and linear autoencoders, employing the overnight index swap (OIS) rates for GBP (SONIA) as the dataset.