PurgeBench: A Benchmark and Metric for Adversarial Memory-Poison Recovery in LLM Agents

Research on LLM-agent memory poisoning has concentrated on the attack and prevention stages; the recovery stage - detecting a compromise, removing the poison, and restoring a trustworthy memory store - is measured by no existing benchmark. This work specifies PurgeBench, a benchmark for adversarial memory-poison recovery, and Recovery-Completeness (RC) = HarmonicMean(ER, UR) x RR over eradication (ER), utility retention (UR), and re-emergence resistance (RR). Both degenerate procedures - doing nothing and wiping the store - score zero by construction. v0.3 runs a reproducible reference implementation over the memorywire memory substrate with 30 poisons across five classes and 20 benign checks, scoring seven recovery procedures including the real OWASP Agent Memory Guard. Findings: no procedure exceeds RC ~= 0.64; provenance-based forget (memorywire per-entry source) is the strongest lever; the entangled poison class - a directive hidden inside a trusted memory - defeats every method (the open problem); and content-anomaly detection (Agent Memory Guard, content-detector mode) scores near zero on semantic poison, indicating provenance and rollback - not content anomaly detection - are the levers that work. Code: https://github.com/mthamil107/purgebench