From Models to Systems: A Comprehensive Survey of Efficient Multimodal Learning

Read full text
Authors: Pan Wang, Siwei Song, Hui Ji, Siqi Cao, Heng Yu, Zhijian Liu, Huanrui Yang, Yingyan (Celine) Lin, Beidi Chen, Mohit Bansal, Xiaoming Liu, Pengfei Zhou
+3 more Ming-Hsuan Yang, Tianlong Chen, Jingtong Hu
Cited by: 1

Abstract

The escalating scale of multimodal models has exposed critical bottlenecks in computation, memory, and deployment, establishing Efficient Multimodal Learning (EML) as a distinct research frontier. Despite rapid progress, a holistic understanding of what, how, and where efficiency emerges across the multimodal learning stack remains elusive. This survey bridges this gap by providing the first structured, model-to-system taxonomy of efficiency in multimodal intelligence. We synthesize over 280 studies into three foundational levels-- model , algorithm , and system --each targeting a distinct optimization axis. The model level focuses on architectural efficiency, encompassing modality-specific and unified encoders, structural sparsity, and modular adapters. The algorithm level refines execution through token compression, pruning, quantization, knowledge distillation, speculative decoding, cache reuse, and runtime sparsity, balancing computational cost with alignment fidelity. The system level extends efficiency to real-world deployment via memory management and serving, edge-cloud collaboration, latency-aware scheduling, and hardware-software co-design. We further analyze Efficient Multimodal Large Language Models (MLLMs) as an integrative case study, demonstrating how these three layers coalesce into adaptive, resource-aware reasoning frameworks. Finally, we discuss emerging applications and open challenges-including unified tokenization, generalization and robustness across modalities, human-and hardware-aware adaptation, and privacy constraints. This survey establishes a coherent foundation for designing multimodal systems that are not only capable and generalizable but also efficient, adaptive, and deployable at scale.

Fields of Study & Subjects

Speech and dialogue systems Multimodal Machine Learning Applications Domain Adaptation and Few-Shot Learning

Publication details

Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Publication date
26 January 2026
Document type
Preprint