Exact feature collisions in neural networks

Predictions made by deep neural networks have been shown to be highly sensitive to small changes made in the input space where such maliciously crafted data points containing small perturbations are being referred to as adversarial examples. On the other hand, recent research suggests that the same networks can also be extremely insensitive to changes of large magnitude, where predictions of two largely different data points are mapped to approximately the same output. In such cases, features of two data points are said to approximately collide, thus leading to largely similar predictions. Our results improve and extend prior work on approximate feature collisions in neural networks and provide specific criteria for data points to have colliding features from the perspective of weights of neural networks, revealing that neural networks (theoretically) not only suffer from features that approximately collide but also suffer from features that exactly collide. We identify sufficient conditions for the existence of such scenarios, hereby investigating a large number of deep neural networks that have been used to solve various computer vision problems. Furthermore, we propose the Null-space search, a numerical approach that does not rely on heuristics, to create data points with colliding features for any input and for any task, including, but not limited to, classification, segmentation, and localization.