FedSame: A Bayesian Similarity-Aware Framework for Federated Multitask Learning

Accurate dynamic modeling of task correlations is crucial for enhancing collaborative efficiency and personalized performance in federated multi-task learning (FML), yet existing approaches struggle with heterogeneous environments due to static assumptions or implicit modeling. Moreover, task relationships typically remain implicit, embedded within data distributions and parameter variations, making precise modeling inherently challenging. This challenge is further intensified in Non-IID settings, where data heterogeneity impedes both the identification and accurate estimation of task relationships. To address these challenges, we propose FedSame, a similarity-aware FML framework that leverages Bayesian inference to dynamically model inter-task relationships. The key innovation of FedSame lies in its probabilistic reformulation of task relationship modeling, where an adaptive similarity matrix undergoes continuous Bayesian updates to precisely track evolving task relationships. FedSame’s technical core combines Beta distribution priors with Bayesian update rules, enabling fine-grained detection of subtle variations in task relationship variations during training, and computationally efficient dynamic updates by leveraging the conjugacy property of the Beta distribution. Extensive experiments on two real datasets and a synthetic dataset demonstrate that FedSame consistently outperforms five state-of-the-art baselines in both task relationship modeling accuracy and multi-task classification performance. Remarkably, FedSame attains 73% accuracy in multi-attribute classification on CelebA and 85% accuracy in task relationship modeling on synthetic data, all while maintaining robust performance and notable adaptability in heterogeneous federated environments.