Randomness is good enough in DAG-based Federated Learning

Directed Acyclic Graph (DAG) Federated learning was imposed recently as a promising alternative for federated learning in order to cope with environments prone to asynchrony and heterogeneity. Strongly inspired by the advances in blockchain systems dedicated to IoT environments (e.g. IOTA), DAG-based federated learning maintains at each node in the network a DAG structure which encodes the history of trained models in the network and their training dependencies. A node generates a new model by selecting in a random manner input models in its local DAG, aggregates them, retrains, and sends the result of training to the other nodes in the network. The contribution of this paper is threefold. We conduct an in-depth experimental analysis of a critical hyperparameter-the number of random selected tips-to quantify its impact on the final accuracy and convergence of the global model. Furthermore, we formalize the fairness of training and propose a novel fair tips selection strategy. We extensively evaluate the DAG-based federated learning enriched with both random and fair selection strategies versus classical centralized federated learning, asynchronous federated learning (the extension of classical federated learning to asynchronous and heterogeneous environments), and fully centralized sequential learning. In our experiments, we capture the heterogeneity and asynchrony of the network by introducing a new parameter, drop-off, that can be of independent interest.