A study of video compression and its impact on video LLMs in open-ended VideoQA

Large Language Models (LLMs) have recently been extended to the visual domain and have achieved significant progress from images to videos. Video LLMs have demonstrated strong performance in video understanding tasks such as Video Question Answering (VideoQA). However, these models typically operate on compressed videos generated by video coding standards for efficient storage and transmission. Such compression inevitably introduces distortions and may expose vulnerabilities in video LLMs. Despite this, the effect of compression on video LLMs has not been systematically examined. In this study, we analyze the impact of video compression on the performance of VideoLLaMA3 by applying various compression configurations to the MFQE v2.0 dataset. Additionally, we conduct open-ended VideoQA using manually constructed question–answer pairs. Our results show that stronger compression consistently degrades performance, which indicates the need for compression-aware evaluation and the development of more robust video LLMs.