On the Convergence of HalpernSGD

We study HalpernSGD for minimizing a convex Fréchet differentiable function with Lipschitz gradient on a real Hilbert space. Its deterministic orbit converges strongly to the anchor-selected minimizer \(P_S(u)\) under vanishing steps that need not be square summable. For stochastic oracles, weighted summability of the scaled martingale fluctuation and the \(\F_n\)-measurable bias yields almost sure strong convergence, while expected last-iterate bounds require suffix regularity of the error budget. In the unbiased case, for the critical choice \(\varepsilon_n=n^{-1/2}\), \(α_n=(\log n)/n\) eventually, a uniform conditional moment bound of order \(p>4\) gives both conclusions. We also discuss finite-difference and zeroth-order applications for which scaled, rather than raw, noise is the natural quantity.