A Nonmonotone Adaptive Trust Region Method Based on Conic Model for Unconstrained Optimization

We propose a nonmonotone adaptive trust regionmethod for unconstrained optimization problemswhich combines a conicmodel and a new update rule for adjusting the trust region radius. Unlike the traditional adaptive trust region methods, the subproblem of the new method is the conic minimization subproblem. Moreover, at each iteration, we use the last and the current iterative information to define a suitable initial trust region radius. The global and superlinear convergence properties of the proposed method are established under reasonable conditions. Numerical results show that the new method is efficient and attractive for unconstrained optimization problems.


Introduction
In this paper, we consider the following unconstrained optimization problem: where :   →  is a continuously differentiable function.
Trust region method is effective for solving (1).In 1970s, Powell [1] established the convergence result of trust region method.Yuan [2], Nocedal, and Yuan [3] proposed various trust region methods for optimization problems.It calculates a trial step by solving the subproblem min   () =     + where   = ∇  (  ) is the gradient of the objective function at   ,   ∈  × is a symmetric matrix which is either the Hessian matrix of  at   or an approximation to it, ‖ ⋅ ‖ refers to the Euclidean norm, and Δ  > 0 is the radius of an area, called the trust region, where the model is trusted.Let   be the solution of (2).The actual reduction of the objective function is defined by and the predicted reduction of () is defined by The ratio between these two reductions is defined by and it is normally used to test whether the trial step   is accepted or the trust region radius needs to be adjusted.The next iterate  +1 is chosen by the following formula: where  0 ∈ [0, 1) is a constant.The next trust region radius is chosen as where 0 ≤  1 <  2 < 1, 0 <  0 < 1 <  1 are constants.
In comparison with quasi-Newton methods, trust region methods converge to a point which not only is a stationary point but also satisfies second-order necessary conditions.
Sartenaer [17] presented a strategy for determining automatically an initial trust region radius.Fan and Yuan [18] proposed a trust region method with the trust region radius converging to zero.Recently, Zhang et al. [19] gave an adaptive trust region method, and they solve the subproblem (2) with and  is a nonnegative integer.B is a safely positive definite matrix based on Schnabal and Eskow modified Cholesky factorization.Therefore, instead of adjusting Δ  , one adjusts  at each iteration.Based on [19] and a simple subproblem model, Sang and Sun [20] proposed another adaptive trust region method with line search.The numerical results show that the adaptive trust region algorithms [21][22][23][24] are more effective than traditional trust region methods for unconstrained optimization problems.
The adaptive trust region methods listed above are based on quadratic model; however, when the objective function has strong nonquadratic behavior, the quadratic model methods often produce a poor prediction of the minimizer of the function.In 1980, Davidon [25] proposed the conic model methods for unconstrained optimization problems.A typical conic trust region subproblem is as follows: where   () is called conic model which is an approximation to (  +   ) − (  ).The vector   is the associated vector for the collinear scaling in the th iteration, and it is normally called the horizontal vector.If   = 0, the conic model reduces to a quadratic model.Also, one can see that   () is quadratic along any direction  ∈   satisfying     = 0. Recently, trust region methods based on conic model have been presented and studied; see [26][27][28][29][30][31][32].Numerical results show the efficiency of the conic trust region methods.
In the traditional trust region methods, the next trust region radius is updated by the current ratio   , which is reasonable if the matrix   is exactly as the Hessian   and if the trust region subproblem (2) is solved exactly.However, in practical computations, the matrix   is often obtained approximately and the subproblem is solved approximately.In such a case, it may be more reasonable to adjust the next trust region Δ +1 according to not only   but also the previous ratios { − , . . .,   }, where  is some nonnegative integer.
In order to measure the agreement between the model function and the objective function, we define the following ratio: where   ∈ [0, 1] is the weight of  − , such that min{,} In this paper, we propose a new adaptive trust region method which combines the conic model and nonmonotone technique which was given in [33] and a new update rule for adjusting the trust region radius.Our method is different from other adaptive trust region methods in three points.First, the subproblem is conic model which can produce a better prediction of the minimizer of the function than quadratic model methods when the objective function has strong nonquadratic behavior.Second, at each iteration, our method generates a suitable trust region radius automatically based on the current and last iterative information.Last, we use a new update rule (8) to adjust the next trust region radius.The new trust region model is more consistent with the objective function at the current iterative point.According to the nonmonotone technique in [33], our method constructs an iterative series {  } such that the sequence {(  )} is not monotonic, which can accelerate the convergence rate of the minimization process, especially in the narrow curved valley.The global and superlinear convergence properties of the proposed method are proved under reasonable conditions.The numerical results show that the new method is more effective than other trust region methods.
The rest of this paper is organized as follows.In next section, we describe a new nonmonotone adaptive trust region method based on conic model for unconstrained optimization problems.In Sections 3 and 4, we prove the global and superlinear convergence properties of the proposed method under reasonable conditions, respectively.Some numerical results are given in Section 5. Conclusions are summarized in Section 6.

Algorithm Description
In this section, we describe a new nonmonotone adaptive trust region method based on conic model for unconstrained optimization problems.
We obtain the trial step   by solving the following subproblem: where and  is the smallest nonnegative integer such that The new reduction of () is defined by where ∈ (0, 1],  ≥ 1 is a positive integer.We define () = min{ + 1, } and choose The predicted reduction of () is The new ratio   is defined by In what follows, we describe our new nonmonotone adaptive trust region method based on conic model.Algorithm 1 (nonmonotone adaptive trust region method based on conic model).Consider the following.
Step 2. Solve the subproblem (11) and let   be an approximate solution of the subproblem (11).

Remark 2. In Algorithm 1, we define a new reduction of 𝑓(𝑥)
which is different from the actual reduction   − (  +   ) in other methods.Instead of requiring  +1 to be smaller than   , it is only required that  +1 is either less than   or less than the weighted mean of the function values at the last () iterates.
Remark 3. How to choose the horizontal vector   and the approximate Hessian matrix   is one of the crucial issues of a conic model method.More details for generating   and   can be found in [25].If   = 0, Algorithm 1 reduces to the quadratic model trust region method.If we choose  = 1, we obtain a monotonic adaptive trust region method.Hence, Algorithm 1 is a generalization and development of quadratic model trust region method and monotonic trust region method.

Convergence Analysis
In this section we discuss the global convergence of Algorithm 1.Before we address some theoretical issues, we would like to give the following assumptions.
It is similar to Theorem 3.1 in [28], and we can prove the following lemma.Lemma 6. Suppose that Assumption 5 holds.Then, there exists a scalar  ∈ (0, 1) such that This completes the proof.
Lemma 8. Suppose that Assumption 5 holds.Then, Algorithm 1 is well defined; that is, Algorithm 1 can not cycle infinitely in the internal circulation.
Proof.Suppose that Algorithm 1 cycles infinitely many times between Steps 2 and 4 at the current point   .We denote the solution of the conic model (11) by  () and the corresponding predicted reduction by Pred () with the cycling index .Then, we have Hence, the following Pred () ≥  0 (26) holds for  sufficiently large.From   ≥   , we obtain that holds for  sufficiently large, which contradicts with (23).The proof is completed.
In the next lemma, an important decrease property of the function value (  ) is established.Lemma 9. Suppose that Assumption 5 holds and {  } is generated by Algorithm 1.Then, there exists a constant  > 0 such that the following (1) If  = 1, by  0 ≥  0 ,  0 =  0 , and Lemma 6, we have where  =  0 .
Theorem 11.Suppose that Assumption 5 holds.If  = 0, Algorithm 1 either terminates in finite iterations or generates an infinite sequence {  } which satisfies Proof.If Algorithm 1 terminates in finite iterations, the theorem is obviously true.Assume that Algorithm 1 generates an infinite sequence {  } in the following proof.If (34) is not true, then there exists a positive constant  0 , such that           ≥  0 , ∀.

Superlinear Convergence
In general, the superlinear convergence property of Algorithm 1 requires further assumptions, we give the assumptions and the superlinear convergence property in the following theorem.
which implies that the sequence {  } converges to  * superlinear.

Numerical Results
In this section, we compare the performance of Algorithm 1, denoted by NACTR, with NAQTR if   = 0 for all  in Algorithm 1, MACTR if  = 1 in Algorithm 1, CTR method which is a conic trust region algorithm without adaptive technique, and QTR method which is the traditional trust region method without adaptive technique.All programs are written in MATLAB with double precision.For these methods, the trial step   is computed approximately by the algorithm proposed by Lu and Ni in [34] for solving the subproblem (11). 0 is chosen as the identity matrix.The parameters in our algorithm are chosen as  0 = 0.25,  1 = 0.7,  0 = 0.25,  1 = 1.5,   = 1/(), ( = 0, 1, . . ., ()−1),  = 4, and  0 = 0.In our implementation, we prefer the following choice r = 0.85  + 0.15r −1 , where r0 =  0 .The stopping condition is The iteration is also terminated if the number of evaluations exceeds 300.
We provide the results of our tests in Table 1.The numerical results are given in the form of   ,   , where   ,   denote the numbers of iterations and function evaluations, respectively.We chose the test problems from [35].
From Table 1, we can see that, in most of the problems considered, the numbers   and   for the conic model methods are considerably smaller than those required for the quadratic model methods, especially for those functions with strong nonquadratic behavior.This means that the conic model methods are very effective to many unconstrained optimization.Compared with the methods without adaptive technique, the adaptive trust region methods are more attractive.Hence, our new nonmonotone adaptive conic trust region method is an improvement of the existing trust region methods.

Conclusions
In this paper, we propose a new nonmonotone adaptive conic model trust region method for unconstrained optimization.The global and superlinear convergence properties of the new method are proved under reasonable conditions.Our method is attractive in the following aspects.First, the subproblem is conic model which can produce a better prediction of the minimizer of the function than quadratic model methods