SafetyRAG: Anonymous Artifact

Read full text Summarise with VIRAT
Authors: Anonymous
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Anonymous artifact for "SafetyRAG: Policy-as-Data Architecture with Context-Aware Retrieval for LLM Safety" submitted to ASE 2026. Contains pipeline code, PKB construction scripts, experiment configurations, prompt templates, and raw results. --- SafetyRAG v2 — Rebuttal Update. This version supplements the original v1 release (Zenodo record 19237968,https://zenodo.org/records/19237968) with the rebuttal-period experiments.The complete v1 artifact is included unchanged inside this package; if youonly need the original paper's reproduction bundle, please refer to the v1record. SafetyRAG is a Retrieval-Augmented Generation framework that injectsrelevant safety policies into the LLM context at inference time, allowingthe model to refuse policy-violating requests autonomously, without anexternal guardrail classifier. The architectural commitments areDP1 (Policy–Model Separation) and DP2 (Backend Substitutability). What's new in v2: • Paired McNemar + bootstrap 95% CIs on Paper Table 4 raw outputs. SR vs M2 significant at p < 1e-6 on AILuminate and p < 1e-17 on HarmBench; HarmBench discordant pattern 60–0 (SR vs M2) and 104–0 (SR vs M0). • Multi-judge consistency: 4,800 unsafe responses re-judged with Claude Sonnet 4 as an independent second judge. Cohen's κ = 0.630 (substantial), 92.4% concordance. SR > M2 conclusion judge-robust. • Cross-family backend validation on Anthropic Claude Sonnet 4, OpenAI GPT-4o-mini, and xAI Grok-3 (AILuminate + HarmBench). SafetyRAG transfers without modification (DP2). • Scale test: 12 → 200 synthetic policies; TF-IDF retrieval latency essentially flat (0.59 → 0.61 ms/query). Linear extrapolation: 0.76 ms at N=1,000, 2.52 ms at N=10,000 — under 1% of LLM latency at any realistic scale. • Dynamic Policy Update: three PKB modifications (remove / add / modify) on the original Llama-3.2-3B backend with src_lines_changed = 0, PKB JSON write < 0.4 ms, FAISS rebuild 4.3–5.7 s, downtime = 0. DP1 validated end-to-end. Anonymization: All response files replace the raw benchmark prompt witha SHA-256 hash (prompt_hash) per MLCommons AILuminate license. No APIkeys are embedded anywhere in the artifact (verified by automated scanbefore packaging). See README.md and CHANGELOG.md inside the zip forfull documentation.

Fields of Study & Subjects

Adversarial Robustness in Machine Learning Ethics and Social Impacts of AI Hate Speech and Cyberbullying Detection

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
26 March 2026
Language
EN