Visual-language foundation modelsandartificial intelligence agents in healthcare: Bridging from technological innovation to clinical impact

Read full text
Authors: Abdulrhman Hussein Al-Jebrni, Zehua Jiang, Yiming Qin, Yue Qiu, Nan Jiang, G Li, Y Zheng, T WONG, Suhyeon Kim, Younhyun Jung, Dian Zeng, Bin Sheng
Cited by: 6

Abstract

Recent advances in foundation models (FMs) have enabled artificial intelligence systems to acquire generalizable capabilities, with language and visual models demonstrating adaptability across diverse healthcare applications and multimodal vision-language FMs (VLFMs) supporting complex tasks such as report generation and question answering. Building on these developments, autonomous agents, particularly vision language agents (VLAs), represent the next frontier, extending AI from perception and recognition to cognition, decision-making, and action. By integrating multimodal understanding with planning, interaction, and tool use, VLAs introduce autonomous intelligence capable of managing multi-step clinical workflows, adapting over time, and collaborating with clinicians. This survey provides a comprehensive perspective on VLAs in healthcare. We examine their current progress, analyze the major challenges hindering their integration, and highlight promising directions for future development. Unlike prior surveys that primarily focus on diverse FMs, our work prioritizes emerging VLFMs and incorporates agentic concepts, emphasizing their transformative potential beyond narrow research prototypes. Specifically, we (i) evaluate VLAs in terms of VLFM architectures and adaptation strategies and agentic design, (ii) assess their potential impact across diverse clinical workflows, and (iii) discuss pathways towards next-generation clinical decision support, considering technical and clinical challenges.

Fields of Study & Subjects

Artificial Intelligence in Healthcare and Education Machine Learning in Healthcare Electronic Health Records Systems

Publication details

Journal / source
Computational Visual Media
Publisher
Tsinghua University Press
Pages
1–38
Publication date
1 January 2026