DeHub: Learning Hub-Resistant Representations for Text-Video Retrieval

Text-video retrieval has achieved remarkable progress through fine-grained cross-modal alignment. However, the hubness problem, where a small subset of samples disproportionately dominate nearest-neighbor rankings for many queries, remains a fundamental obstacle in high-dimensional embedding spaces. Existing approaches primarily rely on post-hoc normalization during inference, making them sensitive to prior data distributions and vulnerable under domain shift. To address this limitation, we propose DeHub, a training-phase framework that learns hub-resistant representations by explicitly suppressing hubness during optimization through a dual-stage design. In the deterministic stage, a Hierarchical Subspace Decomposition (HiSD) module performs in-batch PCA to extract latent semantic subspaces, enriching representation granularity and reducing embedding collapse toward central hubs. A hubness weighting loss further identifies high-centrality samples and adaptively increases their contrastive penalties, mitigating semantically relevant hubs. In the probabilistic stage, text-video pairs are modeled as Gaussian distributions rather than fixed points, and a boundary distance loss enforces extremum-based separation at distributional boundaries, effectively repelling semantically irrelevant hubs. Extensive experiments on five benchmarks demonstrate consistent improvements over state-of-the-art methods. Moreover, cross-domain evaluations confirm that training-phase hubness suppression leads to stronger generalization under distribution shift.