AV-IO: Audio-Visual Indoor-Outdoor Classification Dataset via Multimodal Weak Supervision

Indoor-Outdoor classification is traditionally studied with vision cues alone. However, humans naturally combine auditory context to resolve ambiguity (e.g., traffic noise indoors, silent outdoor scenes). Motivated by this observation, we study the problem of multimodal indoor/outdoor classification. Because no public dataset provides indoor/outdoor labels for Audio-Visual (AV) data, we construct a pseudo-labeled dataset using large multimodal teachers (CLIP for vision, CLAP for audio). Our training and evaluation dataset is curated from the TAU Urban Acoustic Scenes dataset (train/val/test folds), and for cross-dataset validation, we use the AVE dataset (test fold). As the inference times of these recent Vision Language Models (VLMs) are not feasible for real-time deployment on any edge devices, a lightweight student network trained with the knowledge acquired through CLIP/CLAP models (teacher) is proposed. Our experiments establish a benchmark on this dataset for indoor outdoor classification task, and ablations show that audio improves over a vision-only baseline, but the gain is modest. Interestingly, the proposed solution remains functional even when the audio modality is absent. Overall, we demonstrate the feasibility of AV Indoor-Outdoor (AV-IO) classification using weak supervision from foundation models, and release the first baseline results on this setting. The codes and the dataset will be made public.