Decomposed Trust: Privacy, Adversarial Robustness, Ethics, and Fairness in Low-Rank LLMs

Large language models (LLMs) have driven major advances across domains, yet their massive size hinders deployment in resourceconstrained settings.Low-rank factorization addresses this challenge by compressing models to effectively reduce their computation and memory consumption while maintaining accuracy.While these compressed models boast benign performance and system-level advantages, their trustworthiness implications remain poorly understood.In this paper, we present the first comprehensive study of how low-rank factorization affects LLM trustworthiness across privacy, adversarial robustness, ethics, and fairness, complemented by an explainabilitydriven analysis of the internal mechanisms behind these trust-related changes.We evaluate multiple LLMs of different sizes and architectures compressed with various lowrank factorization algorithms, revealing key insights: 1 low-rank factorization preserves training data privacy but weakens the protection of personally identifiable information during conversations; 2 adversarial robustness is generally enhanced under compression; 3 ethics degrades in zero-shot prompting but partially recovers in few-shot prompting; 4 fairness declines under compression.Beyond compression, we investigate how model scale and finetuning affect trustworthiness.Additionally, to move beyond black-box analysis, we employ a gradient-based attribution to identify which layers of LLMs contribute most to adversarial robustness.