Performance Modeling of Lightweight Retrieval-Augmented Large Language Models for Low-Resource Plastic Surgery Settings

Read full text Summarise with VIRAT
Authors: Nora Yanyi Sun, Ariana Genovese, Srinivasagam Prabha, Cesar Abraham Gomez-Cabello, Syed Ali Haider, Bernardo Gabriele Collaço, Theophilus Pan, Nadia G. Wood, Antonio J. Forte
Cited by: 0
References: 40

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Background: Large language models (LLMs) are being used by surgeons for education and reference yet concerns about hallucinations and reliability limit safe adoption. Retrieval-augmented generation (RAG) can offer a potential solution by grounding responses in a high-quality external database (e.g., medical textbooks) to enhance accuracy. However, performance tradeoffs across different RAG configurations—many of which exponentially increase computational cost—remain poorly characterized. Methods: In total, 120 lightweight, open-source RAG configurations were evaluated across 40 plastic surgery-focused question-answering tasks (20 single-hop, 20 multi-hop), spanning multiple subspecialties (4800 total evaluations). Configurations varied by base LLM (Phi-3-mini-128k-instruct vs. BioMistral-7B), embedding model, database size, chunk size, and query hop type. Performance was assessed using semantic similarity (Ragas) to physician-validated reference answers. Performance was analyzed using linear mixed-effects regression with query as a random effect and fixed and interaction effects selected via likelihood testing and AIC. Results: High performance was achievable using lightweight, open-source models. While BioMistral-7B had high mean sematic similarity under specific configurations (mean semantic similarity up to 0.786), Phi-3-mini-128k-instruct demonstrated more consistent performance across query complexity. Larger database sizes significantly improved semantic similarity, with the largest gain at intermediate sizes (e.g., size 5: +0.043, p = 0.001). Embedding choice had a strong effect, with bge-large-en-v1.5 improving performance (p = 0.0016) and Bio_ClinicalBERT markedly reducing it (p < 0.001). Multi-hop queries substantially reduced performance (p < 0.001), though this effect was attenuated for Phi-3-mini-128k-instruct via a strong model × hop-type interaction (p < 0.001). Conclusions: RAG systems for plastic surgery do not require large proprietary models, as performance depends on configuration choices and interaction effects rather than isolated components. With advancements, predictive modeling may enable resource-efficient, safe deployment of clinical RAG systems.

Fields of Study & Subjects

Artificial Intelligence in Healthcare and Education Topic Modeling Information Retrieval and Search Behavior

Publication details

Journal / source
Bioengineering
Volume
13
Issue
4
Pages
378–378
Publication date
25 March 2026
Language
EN