Consistency Should Be the Priority for Unified Multimodal Models

Read full text
Authors: Zhaolong Su, Yinyi Luo, Yiqiao Jin, Mengqi Zhang, Wenyue Hua, Srijan Kumar, Qingsong Wen, Jindong Wang
Cited by: 0

Abstract

Unified Multimodal Models (UMMs) emerge as a promising paradigm for integrating understanding and generation within a single architecture. This paper argues that current approaches achieve only interface-level unification where inconsistencies are the primary bottleneck. We identify critical dimensions where consistency must be established: (1) Consistency-Aware Asymmetric Grounding, where models must account for uneven or incomplete evidence reliability rather than treating all modalities as equally informative; (2) Consistency-Aware Representation Learning, where the representation trade-off between understanding and generation should be modeled; (3) Consistency-Aware Reasoning, where multi-step and cross-task consistency should be explicitly modeled; (4) Consistency-Aware Post-Training, where specific training should be considered for the conflicting objective for understanding and generation pathways; and (5) Consistency-Aware Evaluation, where benchmarks must shift from measuring isolated capabilities to assessing consistency. We contend that the next leap in unified multimodal intelligence requires resolving these systemic inconsistencies.

Fields of Study & Subjects

Multimodal Machine Learning Applications Domain Adaptation and Few-Shot Learning Speech and dialogue systems

Publication details

Publication date
17 February 2026
Language
EN
Document type
Preprint