Efficient Vector-Multiplicative Privacy-Preserving Retrieval-Augmented Generation for Large Language Models

Retrieval-Augmented Generation (RAG) grounds large language models (LLMs) in external knowledge, yet it is faces a fundamental trade-off between knowledge confidentiality and retrieval efficiency. Existing approaches fail to reconcile this trade-off: i) Cryptography-based solutions (e.g., homomorphic encryption and secure multi-party computation) provide strong privacy guarantees but incur prohibitive computation and communication overheads. ii) Lightweight perturbation-based methods (e.g., differential privacy) offer higher efficiency at the cost of degraded retrieval accuracy or weakened security guarantees. In this paper, we propose CipheRAG, an efficient vector-multiplicative privacy-preserving RAG framework that achieves a principled balance between robust privacy and high-performance retrieval. Technically, we first propose an efficient searchable inner product functional encryption (IPFE) mechanism enhanced with asymmetric locality-sensitive hashing (ALSH), enabling the retrieval of sensitive knowledge while effectively preserving data confidentiality. Secondly, we propose a decryption-enabled attention mechanism that uses linear weights of the attention layer to decrypt knowledge. This mechanism seamlessly integrates decrypted knowledge into the LLM's generation process, achieving efficiency and accuracy. Extensive experiments demonstrate that CipheRAG achieves up to 35x faster generation and 15x faster QKV computation compared to FHE- and OT-based baselines. By avoiding linear retrieval and full-parameter encryption, CipheRAG enables privacy-preserving RAG with bounded computational and communication overheads, making it well-suited for deployment in privacy-sensitive environments.