yshk-mxim/agent-memory: v1.0.1: Agent Memory — Persistent Q4 KV Cache for Multi-Agent LLM Inference on Edge Devices

Agent Memory v1.0.1 Patch release to trigger Zenodo DOI minting. No code changes from v1.0.0. Authors Yakov Pyotr Shkolnikov Abstract Multi-agent LLM systems on edge devices face a memory management problem: device RAM is too small to hold every agent's KV cache simultaneously. On Apple M4 Pro with 10.2 GB of cache budget, only 3 agents fit at 8K context in FP16. A 10-agent workflow must constantly evict and reload caches. Without persistence, every eviction forces a full re-prefill through the model---15.7 seconds per agent at 4K context. We address this by persisting each agent's KV cache to disk in 4-bit quantized format and reloading it directly into the attention layer, eliminating redundant O(n) prefill computation via direct cache restoration. Evaluated on three architectures (Gemma 3 12B, DeepSeek-Coder-V2-Lite 16B, Llama 3.1 8B), cache restoration reduces time-to-first-token by up to 136x. Open-source at https://github.com/yshk-mxim/agent-memory