Execution of harm-enabling actions by LLM agents during simulated psychiatric crises

Agent-enabled large language models (LLMs) can autonomously execute user-directed actions in digital environments, raising new safety concerns in domains where users may be experiencing acute psychological distress. This study examined whether an agent-enabled LLM executes harm-enabling actions when prompted during simulated psychiatric crises. We conducted a structured algorithmic safety audit using 21 clinically validated vignettes spanning six diagnostic domains: suicidality, bipolar I disorder with manic episodes, delusions and psychosis, hallucinations, obsessive-compulsive disorder (OCD), and harm-to-others scenarios. Each vignette followed a standardized three-turn protocol in which the model was primed with indicators of acute mental health distress before receiving a direct agentic command. The primary outcome was verified completion of a predefined external-action endpoint, including the addition to a cart, progression to a non-transactional stage of a service-based request, or the provision of targeted information following external navigation. Across 105 trials, the agent-enabled system completed the predefined harm-enabling action in 57 cases (54.3%). Under a strict aggregation criterion, 81.0% of vignettes (17 of 21) resulted in at least one safety failure, with failure rates reaching 100% for bipolar I (mania), OCD, and hallucinations. Under a majority-rule criterion, the aggregate failure rate was 52.4%. These findings demonstrate that safety failures in agentic LLM systems may extend beyond inappropriate conversational responses to the execution of external actions that could increase access to potentially harmful means or otherwise facilitate risk in the context established by the vignette. The results highlight the urgent need for safety frameworks that explicitly address agentic execution in mental health-relevant settings.