SAFE-AGENT: Runtime Monitoring and Policy Enforcement for Tool-Using LLM Agents in High-Stakes Enterprise Workflows

AbstractTool-using Large Language Model (LLM) agents are increasingly deployed to automate enterprise workflows such as financial reporting, educational assessment, and security operations. While agentic AI improves productivity, it introduces critical cyber security and governance risks, particularly when agents can interact with external tools including web browsers, code interpreters, internal databases, email clients, and enterprise APIs. This preprint proposes SAFE-AGENT, a runtime monitoring and policy enforcement framework designed to reduce tool misuse, confidential data leakage, and unsafe autonomous actions. SAFE-AGENT acts as a control layer between the agent and its tools, validating tool calls against explicit policies, applying data-loss prevention (DLP) checks, assigning risk scores to actions, and generating auditable logs for compliance and incident response. The paper defines a UK-relevant threat model covering prompt injection to tool misuse, indirect prompt injection via retrieved documents, and hallucination-induced unsafe actions. It further proposes benchmark scenarios across finance, education, and SOC workflows, alongside evaluation metrics including Tool Misuse Rate, Data Leakage Incidents, Task Completion Success, and Human Approval Overhead. SAFE-AGENT supports safer enterprise adoption of agentic AI in regulated sectors through a practical, governance-aligned approach.