Competition for attention predicts good-to-bad tipping in AI

Read full text
Authors: Neil F. Johnson, Frank Y. Huo
Cited by: 0

Abstract

More than 50% of the world's population carry devices that can run personal AI companions with no Internet connection and hence limited safety checks. Here, we discuss how a large class of potentially dangerous good-to-bad output tipping in such edge AI can arise at the atomistic scale (i.e., attention head) through competition for the machinery's attention between the conversation's context and competing output basins. We derive an approximate closed-form estimate for the dynamical tipping point denoted as n ∗ here. We present evidence consistent with the predicted immediate versus delayed/no-tipping regime dynamics across seven 124 million–12 billion parameter open-weight models as well as from conversations in independent production-scale studies.

Fields of Study & Subjects

Ethics and Social Impacts of AI Adversarial Robustness in Machine Learning Safety Systems Engineering in Autonomy

Publication details

Journal / source
Patterns
Publication date
1 October 2026
Language
EN