Forward-Propagation-Driven Learning (FPDL): A Unified Theory of Pretraining and Chain-of-Thought Reinforcement

We present a unified theoretical framework that exposes the shared structure of modern large-model training paradigms that depend on the model's own forward-generation distribution to construct learning signals. Examples include autoregressive pretraining (masked / next-token prediction) and reinforcement learning applied to chain-of-thought (CoT) reasoning (RL-CoT). We formalize these approaches as instances of a single family of losses which we call Forward-Propagation-Driven Learning (FPDL). Under FPDL, training updates arise from expectations under the current model distribution pθ, with differing per-token weighting schemes that capture token-level (dense) versus trajectory-level (sparse) optimization pressures. We derive structural equivalences, identify complementary roles of dense pretraining and sparse trajectory-level reward learning, propose analytic measures (optimization density and alignment coefficient) to quantify the trade-off between them, and analyze the induced dynamical systems in parameter space. We provide proof sketches for key propositions, discuss implications for model design and safety-aligned training, and outline empirical protocols and synthetic tasks to validate theoretical predictions. FPDL reframes a broad class of practical training techniques in a common mathematical language, clarifies their complementary strengths, and suggests principled hybridization strategies.