Block ModShift: Model Privacy via Dynamic Designed Shifts

The problem of model privacy against an eavesdropper (Eve) in a distributed learning environment is investigated. The solution is found via evaluating the Fisher Information Matrix (FIM) for the model learning problem for Eve. Through a model shift design process, the eavesdropper’s FIM can be driven to singularity, yielding a provably hard estimation problem for Eve. Both a one-shot and multi-shot solution are designed. These two approaches require the sharing of a modest amount of information with the central server learning the global model. The multi-shot solution has time-varying shifts that prevent Eve from using the temporal correlation of the gradients to learn the shifts. We design a convergence test for Eve to determine if model updates have been tampered with. However, our shift strategies pass the test and thus the shifts are not detectable. The single-shot and multi-shot methods are compared against a noise injection scheme and shown to offer superior performance.