Exploring Performance-Fairness Trade-Offs in Federated Learning

Fairness in federated learning (FL) has emerged as a critical concern, aiming to develop an unbiased model among groups (e.g., male or female) of diverse sensitive features. However, there is a trade-off between model performance and fairness, i.e., improving fairness will decrease performance. Existing approaches have characterized such a trade-off by introducing hyperparameters to quantify client’s preferences over fairness and performance. Nevertheless, these approaches are limited to scenarios where each client has only a single pre-defined preference, and fail to address in cases where each client has multiple preferences. In this work, we aim to design algorithms that allow the trained model to adapt to each client’s diverse preferences in real time. The key challenges lie in (I) associating preferences with the trained model; (II) mitigating data heterogeneity; (III) preventing preference leakage; and (IV) handling data noise. To address these, we propose two preference-aware schemes, PraFFL and PraFFL-R, designed to generate models tailored to specific client preferences. PraFFL tackles challenges (I)–(III) by incorporating a hypernetwork that adaptively adjusts the model according to each client’s preferences, thereby better satisfying individual needs. To further address data noise (challenge (IV)), we introduce PraFFL-R, which enhances the robustness of the learned Pareto front by optimizing worst-case scenarios among preference-specific models. We provide theoretical guarantees demonstrating that PraFFL and PraFFL-R can produce an optimal model customized for any client preference, with proofs of linear convergence in the strongly convex setting and sublinear convergence in the non-convex setting. Experimental results show that our proposed PraFFL and PraFFL-R outperform five fair FL algorithms in terms of the model’s capability of adapting to clients’ different preferences.