DeNAV: Decentralized Self-Supervised Learning with a Training Navigator

Read full text Summarise with VIRAT
Authors: Xuanyu Chen, Nan Yang, Charles Z. Liu, Dong Yuan
Cited by: 0
References: 24

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Federated learning has gained significant attention for enabling effective model training on distributed data. However, in practice, most client data is unlabeled, and reliable communication with a central server is often infeasible at scale. To address this, we consider a more realistic setting where clients hold only unlabeled data and can communicate only with neighbors and propose DeNAV, a novel decentralized self-supervised learning framework for handling such scenarios. DeNAV simultaneously pre-trains multiple lightweight transformer models across clients. To improve training, we design a navigator algorithm to smartly plan the training route of each model and adopt staleness-aware model aggregation to handle the discrepancy of training status between models. DeNAV eliminates the need for server coordination, offering both convergence and consensus guarantees. Extensive experiments show that DeNAV is comparable to state-of-the-art federated self-supervised learning baselines and also surpasses previous decentralized methods with equal communication efficiency.

Fields of Study & Subjects

Privacy-Preserving Technologies in Data Domain Adaptation and Few-Shot Learning Stochastic Gradient Optimization Techniques

Publication details

Journal / source
Machine Learning
Volume
115
Issue
4
Pages
82
Publication date
31 March 2026
Language
EN