A Fine-Grained Model Editing Method Based on the Importance of Parameters

Model editing techniques can modify or update certain knowledge in LLMs at an extremely low cost and thus have achieved significant success. Existing model editing algorithms treat this task as an optimization problem, which can be addressed by the Lagrange multiplier method and by incrementally updating the weights of feed-forward networks (FFNs). However, they simply rely on some analysis to select specific important FFN layers to update these layers' weights, essentially assuming that the remaining unselected layers are unimportant. As a result, the key role that the remaining layers play is effectively ignored. Moreover, existing methods are coarsegrained and also ignore the influence of “unimportant” neurons in the identified “important” layers. By contrast, we propose FGME, a fine-grained model editing method, which not only selects more important neurons rather than layers, but also chooses neurons that have a significant impact on both the original output and the editing target as candidates. Next, we apply our FGME method to existing editing algorithms as a plugin and achieve noticeable improvements. Experiments demonstrate the superiority of FGME and its effectiveness as a plugin method. Our code is available athttps://anonymous.4open.science/r/FGME-220F/.