Higher-order adaptive BFGS algorithm for nonconvex functions

Abstract Quasi-Newton methods are commonly used to solve unconstrained optimization problems, among which BFGS method is one of the most common quasi-Newton method, which has stable and good performance.Whether BFGS method has global convergence or not under weak Wolfe-Powell (WWP) line search is a widely concerned issue in recent years. BFGS method can be easily proved to have global convergence for convex functions, but not for general nonconvex functions. Therefore we propose an modified BFGS algorithm with global convergence, and the following properties are shown in this algorithm:(i) a special descent condition is proposed to classify the iteration points into two categories; (ii) we construct a new paraboloid and the corresponding projection technique by utilizing the information of the third-order expansion of the function; (iii) for the points satisfying the special descent conditions, the original BFGS update formula is used; (iv) for points that do not satisfy the conditions, they are projected onto the proposed paraboloid.Our new improved BFGS algorithm has global convergence under WWP search and also has good numerical performance.