Longitudinal information extraction from clinical notes in rare diseases: an efficient approach with small language models

Abstract Objectives Rare diseases often require longitudinal monitoring to characterise progression, yet much clinical information remains locked in unstructured electronic health records (EHRs). Efficient recovery of such data is critical for accurate prognostic modelling and clinical trial preparation. We aimed to develop and evaluate a small language model (SLM)-based pipeline for extracting longitudinal information from French clinical notes of patients with rare kidney diseases. Methods As a use case, we focused on serum creatinine, a key biomarker of kidney function. We analyzed 81 clinical notes comprising 200 measurements (triplet of date, value and unit). Four open-source SLMs (Mistral-7B, Llama-3.2-3B, Qwen3-4B, Qwen3-8B) were systematically tested with different prompting strategies in French and English. Outputs were post-processed to standardize formats and resolve inconsistencies, and performance was assessed across model size, prompting, language, and robustness to text duplication. Results All SLMs extracted structured triplets, with F1-scores ranging from 0.519 to 0.928 (Qwen3-8B), outperforming the rule-based baseline. Larger models generally performed better, while prompting strategy and language had modest effects across models. SLMs also showed variable robustness to duplicated content common in real-world EHR notes. Discussion Lightweight, locally deployable language models can accurately extract longitudinal biomarkers from unstructured clinical notes. Our findings highlight their practicality for rare diseases where data scarcity often limits task-specific model training. Conclusion SLMs provide a privacy-preserving and resource-efficient solution for recovering longitudinal biomarker trajectories from unstructured notes, offering potential to advance real-world research and patient care in rare kidney diseases. 1) What is already known? Longitudinal monitoring is essential in rare kidney diseases, yet key biomarker data are often locked in unstructured clinical notes. Large language models (LLMs) have shown strong performance in clinical text processing tasks but face major challenges related to privacy, computational cost, and implementation feasibility in healthcare settings. Small language models (SLMs) are emerging as lightweight, locally deployable alternatives whose potential for clinical applications is increasingly recognized. 2) What does this paper add? This study provides the first real-world evaluation of SLMs for extracting longitudinal biomarker measurements in rare kidney disease cohorts. It introduces and validates an efficient extraction pipeline that combines document preselection, SLM prompting, and post-processing to accurately retrieve biomarker measurements from French clinical notes. The findings show that SLM-based extraction can help mitigate data scarcity in rare diseases, thereby improving prognosis modeling and supporting clinical research.