On the Foundations of Reinforcement Learning: Control with Supervised Regression

Reinforcement learning is commonly presented as a third learning paradigm alongside supervised and unsupervised learning, implying a distinct learning mechanism. At the level of execution, however, this classification is misleading. In practical implementations, reinforcement learning applies supervised regression within a control loop, using dynamically constructed proxy targets derived from interaction with an environment. We revisit the foundations of reinforcement learning from an execution-level perspective and show that value functions, policies, and action-value functions are learned through repeated regression on endogenous targets, rather than through the estimation or optimization of any computable stochastic object.Expectation-based formulations commonly used in reinforcement learning presuppose access to transition dynamics and trajectory measures that are not available in model-free settings, rendering these objectives non-executable for the learning system. As a result, optimization in reinforcement learning does not correspond to descent on a stable, task-aligned objective, but proceeds as a guided stochastic search regulated by engineering constraints. This reframing clarifies the role of stabilization heuristics, explains the inherent fragility of model-free reinforcement learning, and highlights how paradigm-level misclassification can obscure the actual mechanisms by which learning systems operate.