Constraint-Driven Evolution of Multimodal Video Intelligence: A Network and System Perspective

Multimodal video intelligence is increasingly shaped not only by algorithmic progress but also by the constraints of networked deployment, such as limited compute and communication resources, real-time latency, energy, and cost. This survey examines the field from a network and system perspective and organizes recent advances into a three-stage, constraint-driven trajectory that links algorithmic capabilities with deployment architectures and network behaviors. The first stage, Perception Enhancement, focuses on real-time closed-loop processing through lightweight models and edge-side control. The second stage, Semantic Understanding, addresses long-sequence and multimodal analysis by leveraging edge–cloud collaboration and distributed scheduling. The third stage, Cognitive Reasoning, supports causal and interactive intelligence via retrieval augmentation, long-term memory, and multi-agent coordination. Across these stages, algorithm–system co-design is reflected in network behaviors such as caching, compression, representation sharing, and cross-modal alignment. Overall, this survey highlights a shift from algorithm-centric design toward network-aware intelligence, from static perception to open-world generalization, and from isolated inference to networked interactive systems, providing insights for bridging AI research with real-world networked applications.