The Global Convergence of a Modified BFGS Method under Inexact Line Search for Nonconvex Functions

Among the quasi-Newton algorithms, the BFGS method is often discussed by related scholars. However, in the case of inexact Wolfe line searches or even exact line search, the global convergence of the BFGS method for nonconvex functions is not still proven. Based on the aforementioned issues, we propose a new quasi-Newton algorithm to obtain a better convergence property; it is designed according to the following essentials: (1) a modified BFGS formula is designed to guarantee that Bk+1 inherits the positive definiteness of Bk; (2) a modified weak Wolfe–Powell line search is recommended; (3) a parabola, which is considered as the projection plane to avoid using the invalid direction, is proposed, and the next point xk+1 is designed by a projection technique; (4) to obtain the global convergence of the proposed algorithm more easily, the projection point is used at all the next iteration points instead of the current modified BFGS update formula; and (5) the global convergence of the given algorithm is established under suitable conditions. Numerical results show that the proposed algorithm is efficient.


Introduction
Consider min f(x)|x ∈ R n , (1) where f: R n ⟶ R and f ∈ C 2 . e multitudinous algorithms for (1) often use the following iterative formula: where x k is the current point, s k � x k+1 − x k � α k d k , α k is a step size, and d k is a search direction at x k . ere exist many algorithms for (1) [1][2][3][4][5][6][7][8][9]. Davidon [10] pointed out that the quasi-Newton method is one of the most effective methods for solving nonlinear optimization problems. e idea of the quasi-Newton method is to use the first derivative to establish an approximate Hessian matrix in many iterations, and the approximation is updated by a low-rank matrix in each iteration. e primary quasi-Newton equation is as follows: B k+1 s k � y k , y k � g k+1 − g k .
(3) e search direction d k of the quasi-Newton method is generated by the following equation: where H 0 is any given n × n symmetric positive-definite matrix, H k � B − 1 k , the Hessian approximation matrix B k is the quasi-Newton update matrix, and g k � g(x k ) is the gradient of f(x) at x k . e BFGS (Broyden [11], Fletcher [12], Goldfarb [13], and Shanno [14]) method is one of the quasi-Newton line search methods and has great numerical stability. e famous BFGS update formula is which is effective for solving (1) [15][16][17][18]. Powell [19] first proved that the BFGS method possesses global convergence for convex functions under Wolfe line search. Some global convergence results for the BFGS method for convex minimization problems can be found in [19][20][21][22][23][24][25][26]. However, Dai [16] proposed a counterexample to illustrate that the standard BFGS method may not be applicable to nonconvex functions with Wolfe line search, and Mascarenhas [27] demonstrated the nonconvergence of the standard BFGS method even with exact line search. To verify the global convergence of the BFGS method for general functions, some modified BFGS methods [28][29][30][31] have been also presented for nonconvex minimization problems. Aiming to obtain a better approximation of the objective function Hessian matrix, Wei et al. [32] proposed a new BFGS method, whose formula is where ) T s k , and the corresponding quasi-Newton equation is as follows: For convex functions, convergence analysis of the new BFGS algorithm was given for weak Wolfe-Powell line search: where δ ∈ (0, 1/2) and σ ∈ (δ, 1). Motivated by the above formula and other observations, Yuan and Wei [33] defined a modified quasi-Newton equation as follows: where y m k � y k + max C k , 0 /‖s k ‖ 2 s k and It is obvious that if C k > 0 holds, then the quasi-Newton method is the method (7); otherwise, it is the standard BFGS method. erefore, when C k > 0 holds, the modified quasi-Newton method (10) and the quasi-Newton method (6) have the same approximation of the Hessian matrix. Inspired by their views, we will demonstrate the global convergence of the modified BFGS (MBFGS) method (10) for nonconvex functions with the modified weak Wolfe-Powell (MWWP) line search [34], whose form is as follows: where δ ∈ (0, 1/2), δ 1 ∈ (δ/2, δ), and σ ∈ (δ, 1). e parameter δ 1 is different from that in paper [34].
is article is organized as follows: Section 2 introduces the motivation and states the given technique and algorithm. In Section 3, we prove the global convergence of the modified BFGS method with MWWP line search under some reasonable conditions. Section 4 reports the results of the numerical experiments to show the performance of the algorithms. e last section presents the conclusion. roughout the article, f(x k ) and f(x k+1 ) are replaced by f k and f k+1 , and g(x k ) and g(x k+1 ) are replaced by g k and g k+1 . ‖ · ‖ denotes the Euclidean norm.

Motivation and Algorithm
e global convergence of the BFGS algorithm has been established for the uniformly convex functions which have many advantages. It is worth considering whether we can use these properties of uniformly convex functions in the BFGS algorithm to obtain global convergence. is idea motivates us to propose a projection technique to acquire better convergence properties of the BFGS algorithm. Given a new numerical formula for (1): where z k is the next point generated by the classical BFGS formula. Moreover, a parabolic form is given as follows: where λ > 2 is a constant. It is not difficult to see that 2 can be considered as the first two terms of the expansion of a quadratic function at z k , whose Hessian matrix is a diagonal matrix with eigenvalue − 2λ. erefore, the BFGS method is globally convergent. By projecting x k onto (14), we obtain the next step x k+1 : e idea of the projection can also be found in [6,8,35]. Based on the above discussions, the modified algorithm is given in Algorithm 1.

Remark 1
(i) x k+1 is the defined projection point in Step 6, and vector y τ k is the same as vector y k in Step 5, where the projection point x k+1 does not work in (10) but does in the next iteration.
Step 5, then the global convergence of the algorithm can be obtained by the modified weak Wolfe-Powell line search, (11) and (12). If not, we can ensure the global convergence of the algorithm using the projection method (15).

Convergence Analysis
In this section, we concentrate on the global convergence of the modified projection BFGS algorithm. e following assumptions are required.

Mathematical Problems in Engineering
Assumption 1 is twice continuously differentiable and bounded from below, and its gradient function g(x) is Lipschitz continuous, that is,  (11) and (12), where α ∈ [p m , p n ], and p n > p m > 0 are constants. Proof.
e detailed proof of the rationality of the line search is given in paper [35]. □ Lemma 1. Let Assumption 1 and g T k d k ≤ 0 hold. If the sequence x k is generated by Algorithm 1, then we have where β > 0 is a constant.
Proof. According to Lemma 1 of paper [35], the following relations are reasonable: (19) and (20), we obtain the following formula with β � min δ, β 1 : Using the definition of y m k , we obtain erefore, s T k y m k ≥ β‖α k d k ‖ 2 . e proof is complete.

Lemma 2.
If the sequence B k is generated by Algorithm 1 and Assumption 1 holds, then the matrix B k is positive definite for all k.
Proof. According to (18), the relation s T k y m k > 0 is valid. us, the proof is complete. □ Lemma 3. If the sequence α k , d k , x k , g k is generated by Algorithm 1 and Assumption 1 holds, then Proof. According (11) and Assumption 1(ii), the following formula obviously exists □ Combining (12) with (16), we obtain Substituting the above inequality into (24), we have (23). e proof is complete.
Step 2: obtain a search direction d k by solving B k d k + g k � 0 Step 3: calculate α k using the inequalities (11) and (12).
Step 4: set w k � x k + α k d k . Step , and go to Step 7; otherwise, go to Step 6.
for at least [t/2] values of k ∈ [1, t] for any positive integer t.

Theorem 2. If the conditions of Lemma 4 hold, then we obtain
Proof. By (23), we can obtain □ en, using Algorithm 1, we have e relationship between (27) and (28) indicates that 4 Mathematical Problems in Engineering which means that ‖d k ‖ ⟶ 0 holds for k ⟶ ∞. Combining Lemma 4 and g k � − B k d k , we obtain is implies that (40) holds. e proof is complete.

Numerical Results
In this section, we perform some numerical experiments to test Algorithm 1 with the modified weak Wolfe-Powell line search and compare its performance with that of the normal BFGS method. We call Algorithm 1 as MBFGS.

General Unconstrained Optimization Problems
Tested problems: the problems are obtained from [37,38]. ere are 74 test questions in total, which are listed in Table 1. Dimensionality: problem instances with 300, 900, and 2700 variables are considered. Himmelblau stop rule [39]: if |f(x k )| > e 1 , then set Image description: Figures 1-3 show the profiles of CPUTime, NI, and NFG. It is easy to see from these figures that the MBFGS method possesses the best performance since its performance curves of CPUTime, NI, and NFG are better than those of the BFGS method. In addition, numerical results of the total CPUTime, NI, and NFG of the modified BFGS method are lower than those of the BFGS method.

e Muskingum Model in Engineering
Problems. e Muskingum model, whose definition is as follows, is presented in this subsection. e key work is to numerically estimate the model using Algorithm 1. Muskingum Model [40]: whose symbolic representation is as follows: x 1 is the storage time constant, x 2 is the weight coefficient, x 3 is an extra parameter, I i is the observed inflow discharge, Q i is the observed outflow discharge, Δt is the time step at time t i (i � 1, 2, . . . , n), and n is the total time. e observed data of the experiment are derived from the process of flood runoff from Chenggouwan and Linqing of Nanyunhe in the Haihe Basin, Tianjin, China. To obtain better numerical results, the initial point x � [0, 1, 1] T and the time step Δt � 12(h) are selected. e specific values of I i and Q i for the years 1960, 1961, and 1964 are stated in article [41]. e test results are listed in Table 3.

Conclusion
is paper gives a modified BFGS method and studies its global convergence under an inexact line search for nonconvex functions. A new algorithm is proposed, which has the following properties: (i) e search direction and its associated step size are accepted if a positive condition holds, and the next iterative point is designed; otherwise, a parabola is introduced, which is regarded as the projection surface to avoid using the failed direction, and the next point x k+1 is designed by a projection technique. (ii) To easily obtain global convergence of the proposed algorithm, the projection point is used at all the next iteration points instead of the current modified BFGS update formula. e global convergence for nonconvex functions and the numerical results of the proposed algorithm indicated that the given method is competitive with other similar methods. As for future work, we have the following points to consider: (a) Is there a new projection technique suitable for the global convergence of the modified BFGS method? (b) Application of the modified BFGS method (10) to other line search techniques should be discussed. (c) Whether the combination of the projection technique mentioned and a conjugate gradient method, especially the PRP method, has good numerical experimental results is worthy of investigation.

Data Availability
All data supporting the findings are included in the paper.

Conflicts of Interest
ere is no conflict of interests regarding the publication of this paper.      Mathematical Problems in Engineering 7