Global Convergence of a Modified Two-Parameter Scaled BFGS Method with Yuan-Wei-Lu Line Search for Unconstrained Optimization

,e BFGS method is one of the most efficient quasi-Newton methods for solving smalland medium-size unconstrained optimization problems. For the sake of exploring its more interesting properties, a modified two-parameter scaled BFGSmethod is stated in this paper. ,e intention of the modified scaled BFGS method is to improve the eigenvalues structure of the BFGS update. In this method, the first two terms and the last term of the standard BFGS update formula are scaled with two different positive parameters, and the new value of yk is given. Meanwhile, Yuan-Wei-Lu line search is also proposed. Under the mentioned line search, the modified two-parameter scaled BFGS method is globally convergent for nonconvex functions.,e extensive numerical experiments show that this form of the scaled BFGS method outperforms the standard BFGS method or some similar scaled methods.


Introduction
where x ∈ R n , and f: R n ⟶ R is a continuously differentiable function bounded from below. e quasi-Newton methods are currently used in countless optimization software for solving unconstrained optimization problems [1][2][3][4][5][6][7][8]. e BFGS method, one of the most efficient quasi-Newton methods, for solving (1) is an iterative method of the following form: where k � 0, 1, 2, . . . , α k , obtained by some line search rule, is a step size, and d k is the BFGS search direction computed by the following equation: where g k � g(x k ) is the gradient of f(x), and the matrix B k is the BFGS approximation to the Hessian ∇ 2 f(x k ), which has the following update formula: where s k � x k+1 − x k and y k � g k+1 − g k . e problems related to the BFGS method have been analyzed and studied by many scholars, and satisfactory conclusions have been drawn [9][10][11][12][13][14][15][16]. In earlier year, Powell [17] first proved the global convergence of the standard BFGS method with inexact Wolfe line search for convex functions. Under the exact line search or some specific inexact line search, the BFGS method has the convergence property for convex minimization problems [18][19][20][21]. By contrast, for nonconvex problems, Mascaren [22] has presented an example to elaborate that the BFGS method and some Broyden-type methods may not be convergent under the exact line search. As such, with the Wolfe line searches, Dai [23] also proved that the BFGS method may fail to converge. To verify the global convergence of the BFGS method for general functions and to obtain a better Hessian approximation matrix of the objective function, Yuan and Wei [24] presented a modified quasi-Newton equation as follows: where y k � y k + max C k , 0 In practice, the standard BFGS method has many qualities worth exploring and can effectively solve a class of unconstrained optimization problems.
Here, two excellent properties of the BFGS method are introduced. One is the self-correcting quality, scilicet; if the current Hessian approximate inverse matrix estimates the curvature of the function incorrectly, then Hessian approximation matrix H k will correct itself within a few steps. e other interesting property is that small eigenvalues are better corrected than large ones [25]. Hence, one can see that, the efficiency of the BFGS algorithm is subject to the eigenvalues structure of the Hessian approximation matrix intensely. To improve the performances of the BFGS method, Oren and Luenberger [26] scaled the Hessian approximation matrix B k , that is, they replaced B k by τ k B k , where τ k is a self-scaling factor. Nocedal and Yuan [27] further studied the self-scaling BFGS method when τ k � (y T k s k /s T k B k s k ). Based on the value of this τ k , Al-Baali [28] introduced a simple modification: τ k � min 1, τ k . e numerical experiments showed that the modified self-scaling BFGS method outperforms the unscaled BFGS method. Many other scaled BFGS methods with better properties will be enumerated. Formula 1. e general one-parameter scaled BFGS updating formula is where c k is a positive parameter, and it is diverse for the selection of the scaled factor c k , which is listed as follows.
Choice A: where the value of c k is given by Yuan [29], and with inexact line search, the global convergence of the scaled BFGS method with c k given by (9) is established for convex functions by Powell [30]. Ulteriorly, for general nonlinear functions, Yuan limited the value range of c k to [0.01, 100] to ensure the positivity of c k under the inexact line search and proved the global convergence of the scaled BFGS method in this form. Choice B: which is obtained as a solution of the problem: min‖s k − c k y k ‖ 2 . e scaled BFGS method based on this value of c k was introduced by Barzilai and Borwein [31] and was deemed the spectral scaled BFGS method. Cheng and Li [32] proved that the spectral scaled BFGS method is globally convergent under Wolfe line search with assuming the convexity of the minimizing function. Choice C: where β k > 0 for k � 0, 1, . . .. Under the Wolfe line search (20) and (21), y T k s k > 0 holds for k � 0, 1, . . ., which implies that c k computed by (11) is bounded away from zero, that is to say, 0 < c k ≤ 1. erefore, in this instance, the large eigenvalues of B k+1 given by (8) are shifted to the left [33]. Formula 2. Proposed by Oren and Luenberger [26], this scaled BFGS method was the single parameter scaled of the first two items of the BFGS update and was defined as where δ k is a positive parameter and is calculated as follows: e parameter δ k assigned by (13) can make the structure of eigenvalue to inverse Hessian approximation more easily analyzed. Consequently, it is regarded as one of the best factors. Formula 3. In this method, the scaled parameters are selected to cluster the eigenvalues of the iteration matrix B k+1 and shift the large eigenvalues to the left. e update formula of the Hessian approximate matrix is computed as where both δ k and c k are positive parameters, and Andrei [34] preset them as the following values: If the scaled parameters are bounded and line search is inexact, then this scaled BFGS algorithm is globally convergent for general functions. A large number of numerical experiments show that the double parameter scaled BFGS method with δ k and c k given by (15) and (16) is more competitive than the standard BFGS method. In this paper, combining (7) and (14), we propose a new update formula of B k+1 listed as follows: where y k is determined by formula (6), Some interesting properties of the BFGS-type method are inseparable from the weak Wolfe-Powell (WWP) line search: where 0 < ξ < σ < 1. ere are many research studies based on this line search [35][36][37][38][39][40][41][42][43].
Our paper is organized as follows. e motivation and algorithm are introduced in the next section. In Section 3, the convergence analysis of the modified two-parameter scaled BFGS method under Yuan-Wei-Lu line search is established. Section 4 is devoted to show the results of numerical experiments. Some conclusions are stated in the last section.

Motivation and Algorithm
Two crucial tools for analyzing properties of the BFGS method are the trace and the determinant of the B k+1 given by (4). us, the corresponding relations are enumerated as follows: Applying the following existing relation in the study of Sun and Yuan [44], where Obviously, the efficiency of the BFGS method depends on the eigenvalues structure of the Hessian approximation matrix, and the BFGS method is actually more affected by large eigenvalues than by small eigenvalues [25,45,46]. It can be seen that the second item on the right side of the formula (25) is negative. erefore, it produces a shift of the eigenvalues of B k+1 to the left. us, the BFGS method can modify large eigenvalues. Moreover, the third term on the right hand side of (25) being positive produces a shift of the eigenvalues of B k+1 to the right. If this term is large, B k+1 may have large eigenvalues too. erefore, the eigenvalues of the B k+1 can be corrected by scaling the corresponding items in (25), which is the main motivation for us to use the scaling BFGS method. In this paper, we scale the first two terms and the last term of the standard BFGS update formula with two different positive parameters and propose a new y k . In subsequent proof, we will propose some lemmas based on these two important tools to analyze the convergence of the modified scaled BFGS method. en, an algorithm framework for solving the problem (1) will be built in Algorithm 1, which can be designed as

Convergence Analysis
In Section 3, the global convergence of Algorithm 1 will be established, and the following assumptions are useful in convergence analysis.
is bounded (ii) e function f(x) is twice continuously differentiable and bounded from below Lemma 1. If B k is the positive definite, c k > 0, and if α k is computed by (22) and (23), then B k+1 given by (17) is an equally positive definite for all k. Proof.
e inequality (22) and (23) indicates that y T k s k > 0. Using the definition of y k , we obtain For any c ≠ 0, where the penultimate inequality follows, and which is obtained by the Cauchy-Schwarz inequality. □ Lemma 2. Let δ k be generated by (16) for k � 0, 1, . . ., then δ k > 0 and inclines to 1.
Step 3: obtain a search direction d k by solving B k d k + g k � 0.
Step 5: find the scaling factors c k and δ k by (18) and (19).

Mathematical Problems in Engineering
In addition, erefore, by Remark 1 and the above inequality, the formula (33) is transformed into which implies (31). From the positive definiteness of B k+1 , for all k sufficiently large.
Proof. Utilizing the identity (26) and taking the determinant on both sides of the formula (14) with c k and δ k computed as in (18) and (16), we have where the penultimate inequality follows Suppose k is sufficiently large, (39) implies (36). e proof is completed.

Theorem 1. If the sequence x k is obtained by Algorithm 1, then
Proof. e proof by contradiction is used to prove (40) holds. Suppose that ‖g k ‖ > F > 0. By Yuan-Wei-Lu line search (22) and f(x) bounded below, we obtain Adding the abovementioned inequalities from k � 0 to ∞ and utilizing Assumption 1 (ii), we have From Assumption 1 (ii) and (42), we have Based on this, given a constant Φ > 0, there is a positive integer k 0 > 0 satisfying Mathematical Problems in Engineering where c is any positive integer, and the first inequality follows the geometric inequality. Moreover, by Lemma 4, we obtain Considering c ⟶ ∞, the above formula and formula (39)

Numerical Results
In this section, numerical results of Algorithm 1 are reported, and the following methods were compared: (i) MTPSBFGS method (B k+1 is updated by (17) with c k and δ k given by (18) and (19)). (ii) SBFGS method (B k+1 is updated by (14) with c k and δ k given by (11) and (16)).
Experiment environment: all programs are written in MATLAB R2014a and run on a PC with an Inter(R) Core(TM) i5-4210U CPU at 1.70 GHz, 8.00 GB of RAM, and the Windows 10 operating system.  better numerical performance between these two methods, that is, the proposed modified scaled BFGS method is reasonable and feasible. e specific reasons for good performance are stated as follows. e parameter scaling the first two terms of the standard BFGS update is determined to cluster the eigenvalues of Muskingum model [50]:    whose symbolic representation is as follows: x 1 is the storage time constant, x 2 is the weight coefficient, x 3 is an extra parameter, I i is the observed inflow discharge, Q i is the observed outflow discharge, n is the total time, and Δt is the time step at time t i (i � 1, 2, . . . , n). e observed data of the experiment are obtained from the process of flood runoff from Chenggouwan and Linqing of Nanyunhe in the Haihe Basin, Tianjin, China. Select the initial point x � [0, 1, 1] T and the time step Δt � 12(h). e concrete values of I i and Q i for the years 1960, 1961, and 1964 are listed in [51]. e test results are presented in Table 7.  Table 7 imply the following three conclusions: (i) based on the Muskingum model, the efficiency of the MTPSBFGS method is wonderful, and numerical performance of these three algorithms is fantastic.
(ii) Compared to other similar methods, the final points (x 1 , x 2 , and x 3 ) of the MTPSBFGS method are competitive. (iii) Due to the endpoints of these three methods being different, the Muskingum model may have more approximation optimum points.

Conclusion
A modified two parameter scaled BFGS method and the Yuan-Wei-Lu line search technology are introduced in this paper. By scaling the first two terms and the third term of the standard BFGS method with different positive parameters, a new two parameter scaled BFGS method is proposed. In this method, the new value of y k is given to guarantee better properties of the new scaled BFGS method. With Yuan-Wei-Lu line search, the proposed BFGS method is globally convergent. Numerical results indicate that the modified two parameter scaled BFGS method outperforms the standard BFGS method and even the same type of the BFGS method. As for the longer-term work, there are several points to consider: (1) are there some new values of c k , δ k , and y k that make the BFGS method based on the update formula (17) perform better? (2) Whether the new scaled method combined with other line search have also great theoretical results. (3) Some new engineering problems based on the BFGS-type method are worth studying.

Data Availability
e data used to support this study are included within this article.

Conflicts of Interest
e authors declare that there are no conflicts of interest regarding the publication of this paper. Mathematical Problems in Engineering 13