Uncertainty quantification in data-centric machine learning

Read full text Summarise with VIRAT
Authors: Julian Martin Rodemann
Cited by: 0
References: 359

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Machine learning and statistics allow for conclusions about something unknown (population) based on limited observations thereof (data) and assumptions thereon (model). These conclusions come with uncertainty, which can originate in any of the three elements: the unknown population itself (hence irreducible uncertainty) or the data and the model (both reducible). Data contributes to this uncertainty in a quantitative and a qualitative way. While the former contribution is self-explanatory and monocausal (too few observations), the latter contribution can be due to complex data collection, pre-processing, merging and the like. Data-centric machine learning refers to methods that account for or directly entail such data selection steps. Here, data is used for two purposes: to draw conclusions about the population in the first place (training) as well as to evaluate these conclusions’ quality later (testing). This cumulative dissertation studies the selection of these two kinds of data: training data (Part III, Contributions 1–9) and testing data (Part IV, Contributions 10–14). By quantifying the involved uncertainty, the dissertation aims to advance the reliability and trustworthiness of data-centric machine learning. As it will turn out, this endeavor requires a closer look at interactions and feedback loops among all three elements from above: population, data and model. The stylized separation among the three will prove illusory. In particular, Part III of the dissertation will demonstrate that various machine learning algorithms let the model self-select the training data, with far-reaching consequences for statistical inference from such data. The dissertation answers the questions of whether and to what degree reliable conclusions about the population are still possible in this scenario. In a similar spirit, Part IV investigates how the selection of testing data for (multicriteria) benchmarking algorithms affects the validity of the benchmarking results. Here, a special emphasis is also put on how this validity depends on multiple criteria and on the way they are aggregated. Part III and Part IV fundamentally rely on decision-theoretic embeddings of training and testing data selection, respectively. All in all, the dissertation offers novel insights into quantifying uncertainty originating from data selection in statistics and machine learning. They lead to more robust and reliable methods as well as critical assessments of existing ones. Both aspects contribute to a safer, more sustainable and less harmful usage of machine learning and statistics.

Fields of Study & Subjects

Machine Learning and Data Classification Machine Learning and Algorithms Statistical Methods and Applications

Publication details

Journal / source
Electronic Theses of LMU Munich (Ludwig-Maximilians-Universität München)
Publication date
1 January 2026
Language
EN
Document type
Thesis