Doctor Sun: a bilingual multimodal large language model for biomedical AI

While biomedical artificial intelligence (AI) shows great clinical potential, existing systems relying on general-domain large language models (LLMs) often lack sufficient medical data to grasp complex healthcare concepts. Furthermore, recent Large Language and Vision Assistant (LLaVA)-based medical multimodal large language models (MLLMs) still struggle to effectively capture the intricate alignments between medical images and text. Therefore, we introduce Doctor Sun, an MLLM specialized in medicine, developed to encode, integrate, and interpret diverse biomedical data modalities such as text and images. In particular, Doctor Sun integrates a pre-trained vision encoder with a medical LLM and conducts two-stage training on various medical datasets, focusing on feature alignment and instruction tuning. Moreover, we release SunMed-VL, a wide-range bilingual medical multimodal dataset, along with all associated models, code, and resources, to freely support the advancement of biomedical multimodal research.