Hidden Good-to-Bad Tipping in ChatGPT-Like AI

Trust in the use of ChatGPT-like Artificial Intelligence (AI) across business, finance, insurance, law, health, and defense sectors is undermined by the fear that its output will tip unnoticed from “good” (e.g., correct, trustworthy, and/or compliant) to “bad,” for example, biased, suboptimal, harmful but with correct syntax, semantics, and plausible logic – hence making it hard for employees, clients, CEOs, etc. to notice and therefore exposing them and their businesses to losses and lawsuits. Here, we identify a deterministic mechanism at the core of all ChatGPT-like systems, which can cause this tipping problem. It suggests that such “good-to-bad” tipping differs fundamentally from the more obvious hallucinations that are known to arise with increased temperature stochasticity during next-token selection. We derive an approximate formula for when this deterministic tipping will occur during an AI’s response to a given prompt, and how it depends on the interactions between the AI’s training, its fine tuning, its architecture, and the composition of the user’s prompt. Our findings could help equip businesses, financial trading entities, insurers, regulators, and operators in any other AI-enabled domains such as law, health, and defense, with new mathematical tools to identify and manage such tipping risks, allowing them to shift toward architectural stress testing and real-time monitoring to perhaps minimize AI-output losses in any real-world setting.