Unleashing the Potential of All Test Samples: Mean-Shift Guided Test-Time Adaptation
Abstract
Visual-language models (VLMs) like CLIP exhibit strong generalization but struggle with distribution shifts at test time. Existing training-free test-time adaptation (TTA) methods operate strictly within CLIP’s original feature space, relying on high-confidence samples while overlooking the potential of low-confidence ones. We propose MS-TTA, a training-free approach that enhances feature representations beyond CLIP’s space using a single-step k-nearest neighbors (kNN) Mean-Shift. By refining all test samples, MS-TTA improves feature compactness and class separability, leading to more stable adaptation. Additionally, a cache of refined embeddings further enhances inference by providing Mean-Shift-enhanced logits. Extensive evaluations on OOD and Cross-Dataset Benchmarks demonstrate that MS-TTA consistently outperforms state-of-the-art training-free TTA methods, achieving robust adaptation without requiring additional training.
Fields of Study & Subjects
Multimodal Machine Learning Applications
Domain Adaptation and Few-Shot Learning
Generative Adversarial Networks and Image Synthesis
Publication details
- Journal / source
- IEEE Transactions on Circuits and Systems for Video Technology
- Publisher
- Institute of Electrical and Electronics Engineers (IEEE)
- Volume
- 36
- Issue
- 8
- Pages
- 11323–11335
- Publication date
- 23 March 2026
- Language
- EN