Parameter-efficient fine-tuning of large language models (LLMs)

Summarise with VIRAT
Authors: Jiaqing Li
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

As the parameter scale of large language models (LLMs) continues to increase, these models have demonstrated remarkable performance across a wide range of natural language processing(NLP) tasks. However, deploying such models in resource-constrained downstream applications poses substantial challenges, primarily due to their enormous parameter counts and the accompanying computational costs. Parameter-efficient fine-tuning (PEFT) has therefore become increasingly important, as it enables models to retain strong performance while significantly reducing training overhead. In this dissertation, we provide a comprehensive overview of PEFT methodologies, with a particular focus on analyzing the underlying principles and mechanisms of a diverse set of representative approaches. Based on these analyses, we propose a structured taxonomy comprising four categories: additive, selective, reparameterized, and hybrid fine-tuning methods. Finally, we perform experiments with several representative PEFT methods to evaluate their practical effectiveness and examine their applications within the field of natural language processing.

Fields of Study & Subjects

Topic Modeling Domain Adaptation and Few-Shot Learning Natural Language Processing Techniques

Publication details

Journal / source
DR-NTU (Nanyang Technological University)
Publication date
1 January 2026
Language
EN
Document type
Thesis