Full-Layer Memory Injection via Compile-Time Key-Value Caches: An External-Knowledge Mechanism for Small-Parameter Language Models
Abstract
Knowledge of large language models (LLMs) is frozen in parameters after pretraining and hard to update at low cost. This paper proposes a full-layer memory injection method based on compile-time key-value (KV) caches. Experiments on Qwen3-0.6B show full-layer token-level KV injection raises private-specification QA score from 0.079 to 0.621, exceeding the text prefix (0.571); sparse or pooled injection degenerates to baseline. Cross-scale validation on Qwen3.8-27B (hybrid, 16/64 full-attention layers) raises 0.133 to 0.642. Full data and code are provided.
Fields of Study & Subjects
Natural Language Processing Techniques
Topic Modeling
Machine Learning in Healthcare
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 5 September 2026
- Document type
- Preprint