Medical-Qwen3: A Robust Medical LLM via Two-Stage LoRA Training and Weighted Adapter Merging

Large language models (LLMs) show strong general capability but often struggle with medical terminology precision and safety-critical instruction following. We present Medical-Qwen3, a 14B-parameter medicaloriented model built on Qwen3-14B through a two-stage LoRA pipeline: (1) domain-adaptive pre-training (PT) to inject broad medical knowledge via continued pre-training (DAPT), and (2) supervised fine-tuning (SFT) to align the model with medical question-answering behaviors through instruction-style data. To balance instruction-following ability and domain knowledge retention, we propose Weighted Adapter Merging, linearly combining SFT and PT adapters before exporting a merged base-model checkpoint. On a held-out medical validation set (F5/F6), the merged model achieves BLEU-4 = 16.38, ROUGE-1 = 20.42, ROUGE-2 = 4.60, and ROUGE-L = 11.54 under a practical decoding configuration (temperature 0.6, top-p 0.8). We further analyze decoding sensitivity and training stability with loss curves and controlled decoding comparisons.