HVD: Human Vision-Driven Video Representation Learning for Text-Video Retrieval

CLIP-based video retrieval methods focus on effective interactions between well-aligned textual and visual features. However, rich visual cues and sparse textual queries pose challenges to precise feature interactions. In this paper, we propose a Human Vision-Driven (HVD) video representation model to overcome this limitation, which consists of two key components: a Frame Feature Selection Module (FFSM) that selects the most relevant frames based on sentence-frame similarity, and a Patch Feature Compression Module (PFCM) that aggregates and re-represents patch features through an attention mechanism. Experiments on five benchmarks demonstrate that HVD achieves SOTA retrieval performance.