Video Language Model Inference Offloading in Cloud-Edge Collaboration: A Decoupled Approach

Recent advances in large language models (LLMs) have extended their capabilities to the visual domain, enabling the understanding of images, videos, and cross-modal content. Unlike traditional LLMs, video language models (VLMs) require not only textual prompts but also the transmission of large-scale video data for inference. Due to their intensive computational and storage demands, these models are typically deployed on cloud servers. However, uploading raw video streams to the cloud incurs substantial transmission latency, which severely limits their applicability in real-time scenarios. To address this challenge, we propose a decoupled parallelism approach for cloud-edge collaborative inference of VLMs. Specifically, we decouple the VLM into a multimodal encoder and a large language model. The video encoder is deployed on edge devices located close to users, while only compact semantic tensors are transmitted to the cloud for subsequent LLM inference. To further optimize communication efficiency, we introduce a redundancy reduction algorithm that eliminates semantically similar frames, substantially shrinking the tensor size while preserving essential spatiotemporal information. We evaluate our framework on the widely used Ultra Video Group (UAG) video dataset, which provides high-quality video sequences suitable for benchmarking real-time video processing systems. Experimental results on benchmark video-language tasks show that our method reduces transmission volume by up to 75 % and achieves nearly a twofold reduction in end-to-end latency compared to cloudonly processing. Importantly, the semantic fidelity of generated responses remains consistently above 90 %, confirming that the proposed framework effectively balances efficiency and accuracy for real-time VLM applications.