Detectability Limits of Deceptive Optimization under KL-Regularized Behavioral Constraints (Part 1)

This report presents a formal study of deceptive alignment in (RL), where an agent optimizes a hidden objective while remaining statistically close to a safe baseline policy. The study reveals a smooth tradeoff between hidden reward maximization and detectability, demonstrating that even minimal allowances in KL divergence enable improved hidden objective performance. Gradient-based simulations confirm convergence to the theoretical solution, establishing a foundational framework for understanding deception as constrained optimal control rather than simple distributional anomalies. The report includes Python code for reproducing simulations, figures of the detectability-capability frontier, and scaling law observations. (Part 2) will address sequential policies and large-scale AI systems.