The Pervasive Blind Spot: Benchmarking VLM Inference Risks on Everyday Personal Videos

Applying Vision-Language Models (VLMs) to pervasive personal videos introduces profound privacy risks. This paper addresses the critical yet unexplored inferential privacy threat, specifically the risk of inferring sensitive personal attributes from seemingly benign data. To address this gap, we crowdsourced a dataset of 508 everyday personal videos from 58 individuals, and benchmarked VLM inference capabilities against human performance. Our findings reveal three key insights: (1) VLMs surpass recruited human evaluators in inferential accuracy, analyzing temporal behavioral patterns rather than relying solely on object recognition. (2) Inferential risk is strongly correlated with specific video characteristics and prompting strategies. (3) VLM-driven explanation towards the inference is unreliable, as we observe a disconnect between the model's reasoning and evidential impact, where ubiquitous objects often serve as misleading confounders.