The measure of deception: an analysis of data forging in machine unlearning

Abstract Motivated by privacy regulations and the need to mitigate the effects of harmful data, machine unlearning seeks to modify trained models so that they effectively ‘forget’ designated data. A key challenge in verifying unlearning is forging—adversarially crafting data that mimic the gradient of a target point, thereby creating the appearance of unlearning without actually removing information. To capture this phenomenon, we consider the collection of data points whose gradients approximate a target gradient within tolerance $\epsilon$, which we call an $\epsilon$-forging set, and develop a framework for its analysis. For linear regression and one-layer neural networks, we show that the Lebesgue measure of this set is small. It scales of the order of $\epsilon$, and when $\epsilon$ is small enough, $\epsilon ^{d}$. More generally, under mild regularity assumptions, we prove that the forging set measure decays as $\epsilon ^{(d-r)/2}$, where $d$ is the data dimension and $r