An entropy-based framework for model aggregation in federated learning

Federated learning is a machine learning technique that enables models to learn while preserving user privacy.In this approach, multiple institutions collaborate to develop a shared model without exchanging raw data.Instead, they share only the model's generated weights.In this article, a novel method for weight aggregation is proposed, based on weighted averages and entropy, within the framework of horizontal federated learning.The aggregation process begins by generating predictions on a validation set.Then, entropy is calculated for the weights from each client, reflecting the uncertainty or variability in their contributions.Finally, a weighted average is applied, and the previously computed entropies are used to determine the influence of each client's weights in the final model.The proposed algorithm has been evaluated on several datasets and compared against widely used methods such as FedAvg, FedProx, and FedOpt.The results indicate that the new approach increased mean accuracy by about 2 percentage points compared to FedAvg.The most significant improvement was observed on the Iris dataset, where accuracy increased by about 6 percentage points.