A Novel Value for the Parameter in the Dai-Liao-Type Conjugate Gradient Method

A new rule for calculating the parameter t involved in each iteration of the MHSDL (Dai-Liao) conjugate gradient (CG) method is presented. The new value of the parameter initiates a more e ﬃ cient and robust variant of the Dai-Liao algorithm. Under proper conditions, theoretical analysis reveals that the proposed method in conjunction with backtracking line search is of global convergence. Numerical experiments are also presented, which con ﬁ rm the in ﬂ uence of the new value of the parameter t on the behavior of the underlying CG optimization method. Numerical comparisons and the analysis of obtained results considering Dolan and Moré ’ s performance pro ﬁ le show better performances of the novel method with respect to all three analyzed characteristics: number of iterative steps, number of function evaluations, and CPU time.


Introduction and Background Results
The topic of our research is solving the unconstrained nonlinear optimization problem where the function f : ℝ n ⟶ ℝ is continuously differentiable and bounded below. Following the standard notation, g k = ∇f ðx k Þ denotes the gradient, s k−1 = x k − x k−1 and y k−1 = g k − g k−1 . Using an extended conjugacy condition Dai and Liao in [1] proposed the conjugate gradient (CG) method where the step size α k is a positive parameter, x k is an already generated point, x k+1 is a new iterative point, and d k is a suitable search direction. The search directions d k are generated by the conceptual formula where the conjugate gradient coefficient β DL k is defined by wherein t > 0 is a scalar. Some well-known formulas for defining β k have been created by modifying the conjugate gradient parameter β DL k [2][3][4][5][6][7][8][9]. One of them is denoted as β MHSDL in [7] by where t > 0 is a scalar as in (5) and d y k−1 = g k − ð∥g k ∥/∥ g k−1 ∥Þg k−1 .
The family of CG methods for nonlinear optimization has reached great popularity lately, thanks to the various benefits and advantages it possesses. The most important property is based on computationally efficient iterations arising from a simple CG rule. This property initiates the high efficiency of CG methods with respect to analogous methods for nonlinear optimization. Moreover, global convergence is ensured under suitable conditions. Finally, the application of various CG methods in solving image restoration problems has become an important research topic [10,11].
Since the parameter t is important for the numerical behavior of Dai-Liao (DL) CG methods [12], one of the most important problems in the implementation of the DL class CG method is to determine a proper value t > 0 which will give desirable results. Many scientists have invested a lot of time and effort in the previous period to determine the best definition of the nonnegative parameter t in the DL class CG methods. So far, the research in finding the appropriate value of t has evolved in two directions. One group of methods is aimed at finding an appropriate fixed value for t [1,2,[6][7][8], while methods from another group promote appropriate rules for computing values of t in each iteration, which ensure a satisfactory decrease of the objective. In our research, we will pay attention to the second research stream: find the parameter t whose values change through iterations so that the faster convergence is achieved. The value of the parameter t defined in the kth iteration will be denoted by t ðkÞ ≔ t k .
In order to complete the presentation, we will restate the main principles proposed so far for computing t k . Hager and Zhang in [13,14] proposed the DL CG method (5), known as CG-DESCENT, where tðkÞ ≡ t k1 is defined by Dai and Kou [15] suggested the conjugate gradient coef- where τ k is the scaling parameter arising from the self-scaling memoryless BFGS method. Clearly, the Dai and Kou (DK) method is a member of the DL class CG methods, which is determined by The results given in [15] confirm that the DK iterations outperform many existing CG methods. Following the development of DL methods, Babaie-Kafaki and Ghanbari [16] defined two new ways to calculate the value of the parameter t in (5), as in the following two formulas: Andrei in [17] proposed the new rule for calculating t in order to define YðtÞ in (5) and defined a new variant of the DL class CG methods, denoted by DLE, with Lotfi and Hosseini in [18] proposed the following rule for determining the parameter tðkÞ, using the expression The backtracking line search.
Journal of Function Spaces and υ > 1/4, C, and r are three positive constants. On the basis of the above overview of the main CG methods and motivated by the strong theoretical properties and computational efficiency of modified Dai-Liao CG methods proposed by many researchers, we suggest a new way of calculating the value of the parameter tðkÞ. As a consequence, the corresponding CG method of DL type, termed as the Effective Dai-Liao (EDL) method, is proposed and its convergence is proven. Numerical testing and comparison with other known DL variants are presented in order to show the effectiveness of the introduced method. Analysis of generated numerical results exhibits that the proposed EDL method is efficient compared with other DL-type methods.
The global organization of sections is described as follows. Introduction, motivation, and a brief overview of the preliminary results are given in Section 1. A new rule for calculating the variable parameter tðkÞ is proposed in Section 2. An effective algorithm and global convergence of the EDL method initiated by tðkÞ are given in the same section. The new EDL method is tested in Section 3 on some unlimited optimization test problems and compared against some known variants of the DL class methods. Finally, concluding remarks are presented in the last concluding section.

A Modified Dai-Liao Method and Its Convergence
Popularity in defining new rules for calculating tðkÞ is a guarantee that such an approach is effective and still insufficiently explored. The idea for defining a new parameter t * k comes from previously described rules for computing tðkÞ, particularly from the paper Li and Ruan [19] and from the idea which can be found in the paper Yuan et al. [11]. Further, analyzing the results from [1, 2, 6-8], we conclude that the scalar t was defined by a fixed value of 0:1 in related numerical experiments. Also, numerical experience related to the fixed valued t = 1 was reported in [1]. According to this experience, our intention is to define variable values tðkÞ inside the interval ð0, 1Þ.
To successfully define tðkÞ with values belonging to the interval ð0, 1Þ, let us start from the definition of the quantity L k which was used in defining the direction d k in [19]. The parameter L k was defined by By putting y * k−1 into L k , the following can be obtained: Further, with certain modifications and substitutions in the equation defining L k , as well as using the function max, which chooses the maximum between the value of the expression d T k−1 g k and 1, we come to a new definition of the parameter tðkÞ. As described in advance imposed desired restrictions, the novel parameter t * k is defined by Effective Dai-Liao (EDL) CG method. Require: An initial point x 0 and quantities 0 < ε < 1, 0 < δ < 1.
: Let k ≔ k + 1, and go to Step 2. It is easy to verify that t * k defined by (16) satisfies Accordingly, t * k ∈ ð0, 1Þ, which was our initial intention. Clearly, greater values of ∥g k ∥ lead to values t * k ↗1. Further, since the trend ∥g k ∥⟶0 is expectable, we can expect smaller values t * k ↘0 in late iterations. Therefore, t * k is suitable for defining corresponding conjugate gradient coefficient YðtÞ or Y 1 ðtÞ and further DL CG iterations (4).
Considering t = t * k in (6), it is reasonable to propose a novel variant of the Dai-Liao CG parameter β EDL k which is subject to the following rule during the iterative process: Before the main algorithm, it is necessary to define the backtracking line search as one of the most popular and practical methods for computing the step length α k in (3). The procedure for the backtracking line search proposed in [20] starts from the initial value α = 1 and generates output values which ensure that the goal function decreases in each iteration. Consequently, it is appropriate to use Algorithm 1, restated from [21], in order to determine the primary step size α k . Algorithm 2 describes a computational framework for the EDL method.
It is necessary to examine the properties of the EDL method and prove its convergence.
(2) The goal function f is continuous and differentiable in a neighborhood P of M with the Lipschitz Assumption 1 initiates the existence of positive constants D and γ satisfying The conditions from Assumption 1 are assumed. In view of the uniform convexity of f , there is a constant θ > 0 that satisfies or equivalently, It follows from (21) and (22) that By (19) and (23), one concludes where the inequality implies θ ≤ L. The inequality (25) initiates Taking into account α k−1 > 0 and the last inequality, we conclude Lemma 2. [22,23]. Let Assumption 1 be accomplished and the points fx k g be generated by the method (3)-(4). Then, it holds Lemma 3. Consider the proposed Dai-Liao CG method, including (3), (4), and (18). If the search procedure guarantees (27), for all k ≥ 0, then the next inequality holds for some 0 ≤ c ≤ 1.
Proof. The inequality (29) will be verified by induction. In the initial situation k = 0, one obtains g T 0 d 0 = −kg 0 k 2 . Since c ≤ 1,   Journal of Function Spaces obviously (29) is satisfied in the basic case. Suppose that (29) is valid for some k ≥ 1. Taking the inner product of both the left-and right-hand sides in (4) with the vector g T k , the following can be obtained: Using (17) in common with (27) and α k−1 > 0, we conclude Now from (30), (31), and it follows that In view of λ ≥ 1, the inequality (29) is satisfied for c = ð1 − ð1/λÞÞ in (33) and arbitrary k ≥ 0.
The global convergence of the proposed EDL method is confirmed by Theorem 4.
Proof. Suppose the opposite, i.e., (34) is not true. This implies the existence of a constant c 1 > 0 such that Squaring both sides of (4) implies Taking into account (18), we can get  We utilized the performance profile given in [26] to compare numerical results for three criteria (NI, NFE, and CPU) generated by five methods (EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6). The upper curve of the selected performance profile corresponds to the method that shows the best performance. Figures 1-3 plot the performance profiles for the numerical values included in Tables 1-3, respectively. Figure 1 presents the performance profiles of the NI criterion generated by the EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 methods. In this figure, it is noticeable that EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 methods solved all tested functions, wherein the EDL method shows the best performances in 57.14% of test functions compared with MHSDL3 (25.00%), MHSDL4 (0.00%), MHSDL5 (0.00%), and MHSDL6 (17.86%). From Figure 1, it is observable that the graph of the EDL method comes first to the top, which means that the EDL outperforms other considered methods with respect to the NI. Figure 2 presents the performance profiles of the NFE of the EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 methods. It is observable that EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 generated solutions to all tested cases, and the EDL method is the best in 67.86% of the functions compared with MHSDL3 (17.86%), MHSDL4 (0.00%), MHSDL5 (0.00%), and MHSDL6 (14.28%). From Figure 2, it is observed that the EDL graph first comes to the top, which confirms that the EDL is the winner with respect to the NFE. Figure 3 contains graphs of the performance profiles corresponding to the CPU time of the EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 methods. It is obvious that EDL, MHSDL3, MHSDL4, MHSDL5, and MHSDL6 solved all tested functions. Further analysis gives that the EDL method is the winner in 67.86% of the test cases compared with MHSDL3 (17.86%), MHSDL4 (0.00%), MHSDL5 (0.00%), and MHSDL6 (14.28%). Figure 3 demonstrates that the graph of the EDL method first comes to level 1, which indicates its superiority with respect to the CPU time.
From the previous analysis of the results shown in Tables 1-3 and Figures 1-3, it can be concluded that the EDL method produces superlative results in terms of all three basic metrics: NI, NFE, and CPU.

Conclusion
A novel rule which determines the value tðkÞ of the parameter t in each iteration of the Dai-Liao-type CG method is presented. The proposed expression for defining tðkÞ is denoted by t * k . Considering t = t * k in (6), a novel variant of the Dai-Liao CG parameter β EDL k is defined and a novel Effective Dai-Liao (EDL) conjugate gradient method is proposed. The convergence of the EDL method is investigated, and the global convergence on a class of uniformly convex functions is established. By numerical testing, we have shown that there is a significant influence of the scalar size of t * k on the convergence speed of the EDL method. Numerical compari-sons on large-scale unconstrained optimization test functions of different structures and complexities confirm the computational efficiency of the algorithm EDL and its superiority over the previously known DL CG variants, such as MHSDL3, MHSDL4, MHSDL5, and MHSDL6. During the testing, we tracked the number of iterations (NI), number of function evaluations (NFE), and spanned processor time (CPU) performances for each function and each method. Analysis of the obtained performance profiles introduced by Dolan and Moré revealed that the EDL method is the most efficient.
We are convinced that the obtained results will be a motivation for further research in defining new values of the parameter t k in the Dai-Liao CG methods. Future research would include research in finding some more efficient rules to calculate the parameter t k during the iterative process. We hope that our proposal of the new expression for defining the parameter t will initiate further research in that direction. It is evident that finding novel approaches in defining different values of t and the conjugate gradient parameter β k is an inexhaustible topic for scientific research, and our approach is only one possible direction in this research.

Data Availability
Data will be provided on request to the first author.

Conflicts of Interest
The authors declare no conflict of interest.