Optimization-driven adaptive FIR filter identification using LMS, gradient descent, and L-BFGS: A comparative performance analysis

This paper describes a comparative study of adaptive finite impulse response (FIR) filter identification through optimization using three algorithms: Least Mean Squares (LMS), batch Gradient Descent (GD), and the Limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) method. A standard simulation approach is introduced to evaluate convergence speed, estimation capability, frequency-response fidelity, and computation cost in the presence of the same signal, noise, and system conditions. The MSE-based identification problem is shaped in a convex quadratic space, which guarantees a stable evaluation of the first-order as well as quasi-Newton optimization principles. Results suggest that LMS has both low-complexity stochastic updates and the worst steady-state error, GD has smoother descent and higher accuracy through much optimization, and L-BFGS has faster convergence due to effective curvature approximation, while also having the most accurate coefficient and frequency-response reconstruction. These quantitative data show remarkable increases in SNR improvement and final MSE for L-BFGS in comparison to the prior mentioned methods. These results illustrate the applicability of quasi-Newton optimization for block-adaptive digital signal processing (DSP) processes with high accuracy, while illuminating the trade-offs associated with real-time and resource-constrained scenarios.