From Models to Systems: A Comprehensive Survey of Efficient Multimodal Learning
Abstract
The escalating scale of multimodal models has exposed critical bottlenecks in computation, memory, and deployment, establishing Efficient Multimodal Learning (EML) as a distinct research frontier. Despite rapid progress, a holistic understanding of what, how, and where efficiency emerges across the multimodal learning stack remains elusive. This survey bridges this gap by providing the first structured, model-to-system taxonomy of efficiency in multimodal intelligence. We synthesize over 280 studies into three foundational levels-- model , algorithm , and system --each targeting a distinct optimization axis. The model level focuses on architectural efficiency, encompassing modality-specific and unified encoders, structural sparsity, and modular adapters. The algorithm level refines execution through token compression, pruning, quantization, knowledge distillation, speculative decoding, cache reuse, and runtime sparsity, balancing computational cost with alignment fidelity. The system level extends efficiency to real-world deployment via memory management and serving, edge-cloud collaboration, latency-aware scheduling, and hardware-software co-design. We further analyze Efficient Multimodal Large Language Models (MLLMs) as an integrative case study, demonstrating how these three layers coalesce into adaptive, resource-aware reasoning frameworks. Finally, we discuss emerging applications and open challenges-including unified tokenization, generalization and robustness across modalities, human-and hardware-aware adaptation, and privacy constraints. This survey establishes a coherent foundation for designing multimodal systems that are not only capable and generalizable but also efficient, adaptive, and deployable at scale.
Fields of Study & Subjects
Speech and dialogue systems
Multimodal Machine Learning Applications
Domain Adaptation and Few-Shot Learning
Publication details
- Publisher
- Institute of Electrical and Electronics Engineers (IEEE)
- Publication date
- 26 January 2026
- Document type
- Preprint