Understanding Data Uncertainty in Land Cover Mapping using Convolutional Neural Networks

In remote sensing, determining land cover from aerial or satellite images plays an important role, as land cover maps are also used as a basis for numerous downstream geodata analyses. Neural networks are increasingly being used as prediction models for processing remote sensing images. These models are capable of assigning each pixel from the images to one class of a set of predefined land cover classes, thereby generating a map. Neural networks such as convolutional neural networks (CNN) enable the processing of large amounts of data and the derivation of application-specific information such as distinguishing between different types of vegetation. The comprehensive expectations for CNN models in the field of land cover mapping include efficient data processing and the development of new analysis options. In this the quality of the map products take a special position. In critical application areas such as infrastructure planning and disaster management a high quality of land cover maps is expected particularly. When determining prediction quality based on test images with known class assignment, the influence of image quality on prediction has not been sufficiently addressed to date. Thus, this work examines the influence of image areas with reduced quality on land cover mapping, for example caused during data collection. Firstly, the various influencing factors on the quality are categorized. Secondly, the image quality is determined, with particular consideration given to the characteristics of aerial images and orthophotos. The determined image quality is then used to identify potential negative influences on the map prediction result. Determining the uncertainty in class assignment is an additional investigated quality feature. Consequently, several quality features are analyzed. The different quality features are represented by quality maps per image and can be compared as such. Considering the learned features of individual CNN model layers in relation to various quality characteristics provides additional insights into the segmentation process. In that respect the modell features show a high-dimensionality. In order to better interpret these relationships, a dimensionality reduction method is used, which transforms the high-dimensional model features into a 2D space and thus enables visualization on dataset level. Based on the conducted analyses of the specific quality maps, there is a overlap between areas with reduced image quality and prediction uncertainty. This correspondence is difficult to represent in quantitative metrics. The automatic image quality assessment highlights significantly more areas with reduced quality than human perception would indicate. Furthermore, prediction uncertainty is influenced by other factors, such as the CNN model. Additionally, a stronger correlation between uncertainties and misclassifications is observed than with image quality. Overall, this study shows which quality analyses at various levels are possible beyond comparing predictions with known class assignments in order to increase confidence in neural networks for land cover mapping.