PolicyGraph: A Policy-Gated MCP Runtime for Safe and Auditable Tool Execution

PolicyGraph is a deliberately narrow governance runtime for Model Context Protocol (MCP) tool orchestration that treats agent execution as governed computation rather than free-form autonomy. The planner LLM is treated as an untrusted proposal generator; the runtime is the authority boundary and fails closed. Key safety invariants (runtime-enforced) • Default-deny tool execution: only tools in the effective allowlist (live discovery ∩ operator allowlist) may run. • Strict JSON-only planning: planner output must be exactly one JSON object; extra text/markdown/fences are rejected. • Schema validation: tool args must match the tool's live MCP inputSchema (required keys present, no extra keys, basic type checks). • Typed output gate: typed payload emitted only if tool returns valid MCP structuredContent, isError != true, and a registered Pydantic model validates it. • Evidence-locked summarization (optional): every summary item must include verbatim evidence copied from canonicalized validated tool output (lexical attribution enforced). Reproducibility & evaluation • Includes LLM-independent gate/property tests (N=1136) covering plan parsing, allowlisting, schema checks, typed parsing, and grounding. • Optional per-run structured traces for auditability (when SAFE_TRACE_DIR is set): query, discovered tools, allowlist, raw planner output, parsed plan, validation decisions, tool I/O, typed payload, and summary. How to reproduce results Run the primary offline evaluation (LLM-independent): • python -m src.eval.offline_gate_eval Outputs: • eval/results/offline_gate_eval.json Scope / non-goals • Single-step runtime: at most one tool call per request. • Default read-only; side-effecting tools are denied by default (toy servers used for evaluation). • Evidence-lock enforces lexical attribution, not semantic entailment/truthfulness. Paper DOI: https://doi.org/10.5281/zenodo.18243637