FiLoRA: Parameter-Efficient Fine-Tuning With Fisher Information-Guided Low-Rank Adaptation

In this letter, we propose a novel parameter-efficient fine-tuning (PEFT) method, namely FiLoRA (Fisher information-guided low-rank adaptation), which incorporates frozen curvature information to guide parameter updates. FiLoRA computes the Fisher information matrix using its Kronecker-factored approximate curvature (K-FAC) from a small set of downstream samples and applies the low-rank approximation to identify the most sensitive parameter update directions. During fine-tuning, parameter updates are constrained within an optimized subspace, and only a compact core matrix is optimized to reduce trainable parameters. Compared with existing PEFT methods, FiLoRA maintains fine-tuning performance while using a significantly smaller number of parameters, providing a geometry-aware solution for fine-tuning pre-trained models.