Building a Domain-Specialized Medical LLM: Pretraining and Multi-Stage Alignment (SFT, PPO, DPO, KTO) for Chinese Medicine

Read full text Summarise with VIRAT
Authors: Junqi Yang
Cited by: 0
References: 3

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

This project studies how far a small medical language model can be improved with a staged alignment pipeline under limited compute. Starting from Qwen3-0.6B, I first contin-ued pretraining on a Traditional Chinese Medicine (TCM) corpus to add domain vocabulary and clinical-style text patterns. I then trained an SFT model on Chinese TCM instruction-response data and built two follow-up branches: an RLHF-style branch with a reward model and PPO, and a preference-only branch with DPO followed by KTO. All models were evaluated on shared category-level scores from MedBench. In my ex-periments, the SFT model remained competitive on factual and reasoning-heavy tasks, while the DPO+KTO branch produced more conservative responses and showed better behavior on open-ended prompts. The PPO branch was harder to stabilize on a small backbone and did not consistently outperform SFT on the categories reported here. These results suggest that for a 0.6B medical model, preference alignment is useful, but its effect depends strongly on how the preference data are constructed and how much capacity the base model has.

Fields of Study & Subjects

Topic Modeling Artificial Intelligence in Healthcare and Education Domain Adaptation and Few-Shot Learning

Publication details

Journal / source
Applied and Computational Engineering
Volume
230
Issue
1
Pages
38–47
Publication date
30 March 2026
Language
EN