Resurfacing the Instance-only Dependent Label Noise Model through Loss Correction

We investigate the label noise problem in supervised binary classification settings and resurface the underutilized instance-only dependent noise model through loss correction.On the one hand, based on risk equivalence, the instance-aware loss correction scheme completes the bridge from empirical noisy risk minimization to true clean risk minimization provided the base loss is classification calibrated (e.g., cross-entropy).On the other hand, the instance-only dependent modeling of the label noise at the core of the correction enables us to estimate a single value per instance instead of a matrix.Furthermore, the estimation of the transition rates becomes a very flexible process, for which we offer several computationally efficient ways.Empirical findings over different dataset domains (image, audio, tabular) with different learners (neural networks, gradient-boosted machines) validate the promised generalization ability of the method.We summarize our main contributions as follows.• We introduce an instance-aware loss correction mechanism based on risk equivalence to fortify a given loss function to combat instance-dependent label noise effectively (theorywise) and efficiently (computation-wise).• We resurface the underutilized instance-only dependent noise model to estimate the transition probability of each instance to complement the surrogate loss function design.We also provide computationally efficient ways to model the transition rates.• Empirically, we validate our approach with moderate and high noise rates over three datasets of different domains with neural networks and tree-based models, showing comparable/better results against the current approaches while being dataset-and machineagnostic.2 RELATED WORK Noise models.Angluin & Laird (1988) formalized the label noise problem and showed learnability under RCN where the transition rate is the same for all x, y pairs.The more realistic CCN model appoints a distinct rate for each possible transition among classes, which is the most widely studied noise model: Liu & Tao (2016) employed data reweighting; Zhang & Sabuncu (2018), Patrini et al. (2017) and Wang et al. (2019) focused on loss correction; and Han et al. (2018), Yu et al. (2019) and Li et al. (2020) introduced effective neural network machineries based on the difference of loss suffered among samples.The more natural noise model where labeling errors depend on the instance itself has also been heavily studied: Xia et al. (2020) and Yang et al. (2022) learn the transition matrix for ILDN via three stage training where the former learns partial matrices to be combined and the latter uses distillation to acquire Bayes-optimal labels; Zhang et al. (2021) and Cheng et al. (2021) progressively correct suspicious data pairs; Bae et al. (2024) perform Dirichletbased resampling using the transition matrix; Du & Cai (2015) reinterpret linear classifiers from a distance-to-boundary perspective under IDN, which Bylander (1998) also studied under monotonic