Efficient Tuning Framework for Resource- Constrained Biomedical Question Answering

Automatic question-answering systems demonstrate valuable utility in the biomedical domain, improving the precision and efficiency of clinical decision-making significantly. Despite large-scale language models achieving notable success in general domains, even outperforming human-level performance in certain aspects, they are still faced with challenges such as data privacy and scarcity in the special domain. This study explores the method for efficient fine-tuning under resource-constrained conditions in the biomedical field. We propose a multi-stage fine-tuning approach that effectively improves the performance of pre-trained language models in biomedical question-answering tasks. Specially, A multi-prompt-based contrastive learning strategy and a multi-prompt self-consistency voting module are introduced, which improve the accuracy of QA tasks. The experiments on the PubMedQA dataset under reasoning-required settings indicate that our approach outperforms domain-specific pre-training models and achieves comparable performance with GPT-4, while the number of fine-tuned parameters is much less than the total parameters of the base model.