Fedhybrid: an asynchronous-synchronous hybrid framework for federated learning

Federated learning architecture inherits the problem that the majority of time of edge nodes is wasted during the waiting phase for obtaining results from server-side. Hence, synchronous and non-parallel algorithms that model the system like an M/G/1 queue is alleviated by splitting the work between server and the edge devices to maximize throughput and minimize delay. But, these parallel algorithms are impacted by past global models and they are affected on convergence. This work presents a hybrid scheme combining synchronous operations to parallel systems. We propose a number-ofserver- communications-based threshold triggered mechanism to suppress the convergence delay. Based on empirical studies on FASHIONMNIST/CIFAR-10/CIFAR-100, we show that our algorithm can achieve fast convergence at the cost of maintaining the same accuracy.