NL2Filter: A Robust CNN Design for Visual Services Over Cloud and Device

Cloud-assisted resource-constrained and widely used outdoor IoT devices deploying highly robust convolutional neural network (CNN) models to provide high-quality visual services have attracted widespread attention from industry and academia. Most existing methods suffer from two limitations: (i) large amount of parameter transmission, and (ii) low robustness in handling data affected by the environment. To this end, this paper proposes a transmission-friendly and robust CNN design method called NonLinear transformation generation Filter with NonLearnable hyperparameters, namely NL2Filter. In NL2Filter, some filters are first designated as seed filters, whose parameters are learnable, that is, updated as the model is trained. Other filters in this layer are generated based on the seed filters and nonlinear transformation function (NLF). The hyperparameters of the NLF in NL2Filter are randomly initialized and remain unchanged, so they can be saved and reproduced using random seed. After the cloud server trains the NL2Filter-CNN model, it only needs to send a small number of learnable parameters and random seed to reproduce the CNN model trained by the cloud server. Compared with sending the complete CNN model, sending only a small number of learnable parameters and random seed can significantly reduce the number of model parameters sent by the cloud server. On the other hand, NL2Filter groups seed filters and then uses NLFs with different hyperparameters to generate diverse filters on demand for each group, thereby improving the model's ability to capture feature diversity without increasing the complexity of the model, thereby improving the model's robustness in processing data affected by the environment. Experimental results on CIFAR-10, CIFAR-10-C, and Icons-50 datasets demonstrate that the proposed NL2Filter outperforms other state-of-the-art methods. Specifically, using the ResNet-101 architecture, on the CIFAR-10-C, NL2Filter's accuracy is about 3.9% higher than that of MonoCNN; On the Icons-50, NL2Filter's accuracy is about 2.7% higher than that of MonoCNN and about 2.3% higher than that of the standard ResNet-101.