Performance of multimodal large language model in understanding digital UML class diagram
Abstract
Integrating multimodal AI and Intelligent Hyperautomation promises to streamline the understanding of visual artifacts like UML diagrams. We systematically evaluate six leading MLLMs—Gemini, GPT, Claude, Mistral, Grok, and Qwen—on extracting classes, attributes, operations, and relationships from digital UML class diagrams. Our analysis reveals significant performance disparities (F1 scores: 32.1–78.4) and a “class-strong, relationship-weak” pattern. Key failure modes include misinterpreting layout conventions and over-relying on textual proximity. These findings expose architectural limitations in structured visual reasoning, providing actionable insights for future model design, prompting strategies, and hybrid human-AI tooling in model-driven engineering.
Fields of Study & Subjects
Speech and dialogue systems
Data Visualization and Analytics
Natural Language Processing Techniques
Publication details
- Journal / source
- Enterprise Information Systems
- Publisher
- Informa UK Limited
- Volume
- 20
- Issue
- 5-6
- Publication date
- 18 February 2026
- Language
- EN