clinical-embedding-fix: Post-hoc corrections for degenerate clinical text embeddings in RAG pipelines

Read full text
Authors: Yngve Mikkelsen
Cited by: 3
References: 3

Abstract

clinical-embedding-fix is a lightweight Python library for diagnosing and correcting degenerate embedding geometry in clinical retrieval-augmented generation (RAG) pipelines without retraining the model. Many transformer-based embedding models produce near-collapsed representations of clinical text, characterised by high pairwise cosine similarity and low effective dimensionality, thereby degrading retrieval accuracy. The library provides geometric diagnostics (participation ratio, average cosine similarity, SVD-based anisotropy), ZCA whitening in both deployment-realistic (corpus-only) and oracle (transductive) modes, and optimal layer selection across transformer hidden states. Validated across 11 model configurations and approximately 1,400 experimental conditions on clinical corpora, corpus-only whitening improved MRR@10 by +0.16 to +0.27 on degraded models. The only dependency is NumPy, and the core intervention can be applied to any existing embedding pipeline in two lines of code.

Fields of Study & Subjects

Machine Learning in Healthcare Topic Modeling Biomedical Text Mining and Ontologies

Publication details

Journal / source
SSRN Electronic Journal
Publisher
Elsevier BV
Publication date
1 January 2026
Document type
Preprint