The Safety Failures We Are Not Instrumenting: A Perspective on Hidden Safety-Critical Challenges in Modern AI Systems

Read full text Summarise with VIRAT
Authors: Enkelejda Kasneci, Gjergji Kasneci
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Current AI safety discourse still focuses disproportionately on visible failures, including obvious harms, dramatic misuse, and hypothetical catastrophic scenarios. This focus is incomplete. In deployed systems, many of the most consequential failures are quieter: plausible rather than spectacular, distributed across components rather than localized in a single output, and normalized by workflows before they are recognized as hazards. We argue that the central safety challenge in modern AI systems is no longer only whether a model emits a harmful response, but whether the broader socio-technical system preserves the conditions under which errors remain visible, contestable, containable, and recoverable. We propose a five-layer framework for diagnosing these hidden risks: (1) epistemic integrity, concerning whether evidence and uncertainty are represented honestly enough to support calibrated reliance; (2) control integrity, concerning whether authority, permissions, and action boundaries remain robust under attack and optimization; (3) temporal integrity, concerning whether safety holds across sessions, memory updates, and deployment drift; (4) organizational integrity, concerning whether institutions retain the capacity to audit, assign responsibility, and intervene effectively; and (5) ecosystem integrity, concerning whether AI systems preserve rather than erode the information environment on which future oversight depends. Across these layers, we identify under-recognized risk patterns, including overreliance, uncertainty and legitimacy laundering in retrieval, prompt injection, reward hacking, memory poisoning, evaluation deception, fictional human oversight, synthetic evidence pollution, and model collapse. We conclude with actionable design and governance recommendations and a research agenda for shifting AI safety from narrow model-centric evaluation toward socio-technical reliability.

Fields of Study & Subjects

Ethics and Social Impacts of AI Safety Systems Engineering in Autonomy Adversarial Robustness in Machine Learning

Publication details

Journal / source
SocArXiv (OSF Preprints)
Publication date
30 March 2026
Language
EN
Document type
Preprint