From Factors to Methods: A Comprehensive Survey on Visual Instruction Tuning Data Selection

The progress of visual-language models has made visual instruction tuning central to multimodal alignment, yet its effectiveness depends heavily on the composition of training data. Visual instruction datasets are often heterogeneous and redundant, necessitating principled data selection to ensure downstream performance. Despite growing interest, prior studies remain fragmented, relying on disparate evaluation criteria and inconsistent terminology that obscure the underlying design space. To address this, we present the first comprehensive survey of visual instruction data selection, providing a unified perspective on both evaluation and selection mechanisms. We introduce a factor-based analytical framework that organizes core data properties into a structured Data Evaluation Factor Library. Furthermore, we categorize existing methods into feature-based, prediction-based, gradient-based, and hybrid paradigms, analyzing how they operationalize scoring and filtering. By bridging evaluation factors with selection mechanisms, this survey consolidates fragmented insights and outlines future directions for data-efficient multimodal learning. To facilitate ongoing research, we maintain an open repository at https://github.com/ ZhuYun97/VIT-Selection-Survey