State of the Art of LLM-Enabled Interaction with Visualization

Read full text
Authors: Mathis Brossier, Tobias Isenberg, Konrad Janek Schönborn, J. Unger, Mario Alfonso Romero, Johanna Björklund, Anders Ynnerman, Lonni Besançon
Cited by: 1

Abstract

We report on a systematic, PRISMA-guided survey of research at the intersection of LLMs and visualization, with a particular focus on visio-verbal interaction -- where verbal and visual modalities converge to support data sense-making. The emergence of Large Language Models (LLMs) has introduced new paradigms for interacting with data visualizations through natural language, leading to intuitive, multimodal, and accessible interfaces. We analyze 48 papers across six dimensions: application domain, visualization task, visualization representation, interaction modality, LLM integration, and system evaluation. Our classification framework maps LLM roles across the visualization pipeline, from data querying and transformation to visualization generation, explanation, and navigation. We highlight emerging design patterns, identify gaps in accessibility and visualization reading, and discuss the limitations of current LLMs in spatial reasoning and contextual grounding. We further reflect on evaluations of combined LLM-visualization systems, highlighting how current research projects tackle this challenge and discuss current gaps in conducting meaningful evaluations of such systems. With our survey we aim to guide future research and system design in LLM-enhanced visualization, supporting broad audiences and intelligent, conversational interfaces.

Fields of Study & Subjects

Data Visualization and Analytics Multimodal Machine Learning Applications Natural Language Processing Techniques

Publication details

Journal / source
HAL (Le Centre pour la Communication Scientifique Directe)
Publication date
21 January 2026
Language
EN
Document type
Preprint