Impact of LLM Scale and Quantization on Information Extraction from Clinical Text
Abstract
Large Language Models (LLMs) show strong potential for extracting structured information from unstructured clinical narratives. However, their adoption in healthcare is constrained by privacy requirements that necessitate local deployment, often under limited computational resources. This study systematically investigates the trade-offs between model size and numerical precision (quantization) in locally deployed, open-source LLMs applied to a clinical information extraction task. Using seven reasoning models from the Qwen3 and DeepSeek-R1 families (0.6B-70B parameters) and six quantization levels (2, 3, 4, 5, 6, and 8 bits), we evaluated performance on the classification of the modified Rankin Scale (mRS) score from 105 neurosurgery patient notes. A linear mixed-effects analysis revealed significant effects for log-scaled model size (β=-0.84, p
Fields of Study & Subjects
Topic Modeling
Artificial Intelligence in Healthcare and Education
Biomedical Text Mining and Ontologies
Publication details
- Journal / source
- Studies in health technology and informatics
- Publisher
- IOS Press
- Volume
- 336
- Pages
- 829–833
- Publication date
- 21 May 2026
- Language
- EN
- Document type
- Book chapter