PARAMETER-EFFICIENT FINE-TUNING FOR MEDICAL TEXT SUMMARIZATION: A COMPARATIVE STUDY OF LORA, PROMPT TUNING, AND FULL FINE-TUNING

Read full text Summarise with VIRAT
Authors: Ulugbek Shernazarov, Rostislav Svitsov, Bin Shi
Cited by: 0
References: 12

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Fine-tuning large language models for domain-specific tasks such as medical text summarization demands substantial computational resources. Parameter-efficient fine-tuning (PEFT) methods offer promising alternatives by updating only a small fraction of parameters. This paper compares three adaptation approaches—Low-Rank Adaptation (LoRA), Prompt Tuning, and Full Fine-Tuning—across the Flan-T5 model family on the PubMed medical summarization dataset. Through experiments with multiple random seeds, we demonstrate that LoRA consistently outperforms full fine-tuning, achieving 43.52±0.18 ROUGE-1 on Flan-T5- Large with only 0.6% trainable parameters compared to 40.67±0.21 for full fine-tuning. Sensitivity analyses examine the impact of LoRA rank and prompt token count. Our findings suggest the low-rank constraint provides beneficial regularization, challenging assumptions about the necessity of full parameter updates.Code is available at https://github.com/eracoding/llm-medical-summarization

Fields of Study & Subjects

Topic Modeling Domain Adaptation and Few-Shot Learning Natural Language Processing Techniques

Publication details

Pages
01–09
Publication date
21 March 2026
Language
EN