Strategically Deceptive Model Deployment in Performative Prediction

Read full text
Authors: Javier Sanguino, Thomas Kehrenberg, Jose Antonio Lozano, Novi Quadrianto
Cited by: 0
References: 30

Abstract

Machine Learning systems are increasingly deployed in decision-making settings that shape user behavior and, in turn, the data on which future decisions are based. Performative Prediction (PP) formalizes this feedback loop by modeling how deployed models induce distributional shifts. It studies how to learn robust and well-performing models under such dynamics. However, existing PP frameworks typically assume that the model governing these decisions is the same model observed by users (therefore, to which they respond). In practice, deployer institutions may instead disclose curated models, while internally relying on distinct opaque models. We introduce Decoupled Performative Prediction (DPP), a framework that explicitly models mismatches between the model governing institutional decisions and the model that shapes user behavior. By analyzing the resulting optimization landscape, we show that DPP admits new different solutions that provably achieve lower risk for the institution than those under classical PP. We further propose an algorithm with provable convergence guarantees under standard assumptions, demonstrating how easy institutions can benefit from strategically deceptive deployment when they control model disclosure and users lack countervailing power. To capture the implications of such behavior, we introduce the deception cost, a quantitative measure of the degree of deception experienced by users. We study settings in which institutions incorporate this cost into the optimization process, motivated by reputational concerns or potential user abandonment, and show that such self-imposed constraints are insufficient to protect users. Overall, our results demonstrate that model disclosure is not merely an ethical consideration but a core technical design decision, underscoring the need for regulations that hold institutions accountable for deceptive deployment practices.

Fields of Study & Subjects

Explainable Artificial Intelligence (XAI) Financial Distress and Bankruptcy Prediction Data Visualization and Analytics

Publication details

Journal / source
Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency
Publisher
ACM
Pages
3190–3216
Publication date
25 June 2026
Language
EN