Enhancing Entity Representations through Soft Prompt Tuning: An Efficient Approach to Biomedical Named Entity Recognition

Summarise with VIRAT
Authors: Alessandro Becci
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Named Entity Recognition (NER) has evolved in recent years from closed-vocabulary systems, constrained by fixed label sets, to zero-shot models capable of identifying entities dynamically through semantic descriptions. Models such as GLiNER exemplify this paradigm shift, making entity recognition generalizable to unseen entity types without additional supervision. Such models use natural language descriptions and Transformer-based architectures to generalize across diverse domains without requiring retraining for new entity types. However, their performance degrades in highly specialized domains such as biomedicine, where complex nomenclature and technical terminology demand domain-specific tuning. While full fine-tuning remains the prevailing method for domain adaptation in NER, the extent to which parameter-efficient fine-tuning (PEFT) techniques can provide comparable specialization has yet to be investigated. We present a comparative analysis of five distinct adaptation techniques applied to GLiNER for biomedical entity recognition: (1) Label Descriptions as Semantic Anchors, a description-driven bi-encoder baseline where entity types are grounded via fine-tuning of the label encoder; (2) Soft Prompting, comprising three sub-approaches: (2.1) Base Approach, using learnable embedding matrices; (2.2) Embedding Injection, inserting trainable prompts into Transformer layers; (2.3) Custom GLiNER Extension, combining Embedding Injection with native fine-tuning; and (3) Baseline Fine-tuning, included as reference method for comparative evaluation. Experiments are conducted on JNLPBA and BC5CDR, two biomedical benchmarks, with evaluation based on Macro and Micro F1 scores. Our Custom GLiNER Extension, training only 13% of model parameters, achieves competitive performance (JNLPBA Macro F1: 0.7016, BC5CDR Macro F1: 0.8526), closely matching standard fine-tuning (0.7164 and 0.8833) at a fraction of the trainable parameters, offering empirical guidance for PEFT approaches.

Fields of Study & Subjects

Biomedical Text Mining and Ontologies Topic Modeling Domain Adaptation and Few-Shot Learning

Publication details

Journal / source
AMS Degree Thesis (University of Bologna)
Publication date
13 March 2026
Language
EN
Document type
Thesis