A Fractional Trust Region Method for Linear Equality Constrained Optimization

A quasi-Newton trust region method with a new fractional model for linearly constrained optimization problems is proposed. We delete linear equality constraints by using null space technique.The fractional trust region subproblem is solved by a simple dogleg method. The global convergence of the proposed algorithm is established and proved. Numerical results for test problems show the efficiency of the trust region method with new fractional model. These results give the base of further research on nonlinear optimization.


Introduction
In this paper, we consider the linear equality constrained optimization problem: min where () are continuously differentiable,  ∈  × , b ∈   , and rank() = .
Trust region methods have the advantage in the theoretical analysis of convergence properties and the practice.Besides, Davidon proposed a conic model which makes use of more incorporate information at each iteration (see [1]).It is believed that combing trust region techniques with a conic model would be appealing.And it has attracted more and more attention of the researchers (see [2][3][4][5][6][7][8]).
For unconstrained optimization problems, we proposed a new fractional model (see [9]): where   ,   ,   ∈   are horizontal vectors,   = ∇(  ),  ∈   ,   ∈  × is symmetric and an approximate Hessian of () at   .Then, the trust region subproblem of the unconstrained optimization problems is min   () , where  0 (0 <  0 < 1) is a sufficiently small positive number, ‖ ⋅ ‖ refers to the Euclidean norm, and Δ  > 0 is a trust region radius.If   =   = 0, then   () is reduced to the conic model.
For linear equality constraints problem, we focus on the problems which are solved by a trust region method with new fractional model.If the constraints are linear inequality constraints or some constraint functions are nonlinear, then a few difficulties may arise.However, for these cases, the linear equality constraints problem may be as the subproblems of them.For example, the inequality constraints can be removed by an active set technique or a barrier transformation, and then the nonlinear constraints are linearized.
In [10], Sun et al. established the algorithm for the problem (1)-( 2) and prove the convergence.However, they do not consider the model computation.In [11], Lu and Ni proposed a trust region method with new conic model for solving (1)-( 2) and carried out the numerical experiments.
In this paper, we use a simple dogleg method to solve fractional model subproblems and present a quasi-Newton trust region algorithm for solving linear equality constrained optimization problems.This is a continuing work of the fractional model (see [9]), where the linear equality constraints (2) are deleted by using null space techniques.
This paper is organized as follows.In Section 2, we give a description of the fractional trust region subproblem.In Section 3, we give a generalized dogleg algorithm for solving the fractional subproblem.In Section 4, we propose a new quasi-Newton method based on the fractional model for solving linearly constrained optimization problems and prove the global convergence of the proposed method under the reasonable assumptions.The numerical results are presented in Section 5.

The Fractional Trust Region Subproblem
In order to solve the problem (1)-( 2), we assume that  is column full rank and constraints are consistent.That is, the current point   always satisfies     = b.Obviously, the constrained condition is equivalent to    = 0 if  =   + .Therefore, combing with (1)-( 4), we can obtain that the trial step   is computed by the following subproblem: It can be found that our trust region subproblem ( 8)-( 11) is the minimization of a fractional function subject to the trust region constraint and the linear constraints.
The null space technology (see [4,12,13]) is an important technique for solving equality constraints programming problems.In the following, we use this technology to eliminate constraint (13).Since  has full column rank, then there exist an orthogonal matrix  ∈  × and a nonsingular upper triangular matrix  ∈  × such that where  1 ∈  × ,  2 ∈  ×(−) , and  1 ∈  × .Then, (13) can be rewritten as Therefore, the feasible point for ( 13) can be presented by for any  ∈  − , where  2  lies in the null space of .Then, the subproblem ( 12)-( 14) becomes min ψ () where ã , b , and c are reduced horizontal vectors, g is the reduced gradient, B is Hessian approximation, and It can be seen that this subproblem has the same form as the subproblem (7) of the unconstrained optimization problems and it can be considered as the subproblem of the unconstrained minimization over .Therefore, we can find a solution of ( 18)-( 19) by the dogleg method.Besides, it is easy to find that ‖‖ = ‖ 2 ‖ ⩽ Δ is equivalent to ‖‖ ⩽ Δ due to   2  2 = .

The Dogleg Method of Fractional Trust Region Subproblem
Now we consider calculating the trial step   of the new subproblem ( 18)-( 19) by a simple dogleg method.Firstly, we need to recall the choice of the Newton point   as the following subalgorithm (see [9]).
In order to discuss the stationary points of ψ(), by computation we have that the derivative of ψ() is where By direct calculating, we have During the analysis, we find that if   ⩾ 0 and c  g ⩽ 0, then (29) and (30) may have no positive real zero points in most of the cases, respectively.Thus, in order to simplify the discussion, we assume In order to discuss the feasible stationary points of   and φ , we first define the sets   and  φ which contain all the positive extremum points of   and φ , respectively, and these extremum points should be inside the feasible region [0,  Δ ].

It is easy to obtain that
where Now, we have the the following conclusions.
Hence, it is easy to get the following theorems.
Therefore, the steepest descent point  cp = − cp  is an approximate solution of the fractional trust region subproblem ( 18)-( 19), where  cp is defined by (43).
Based on the preceding theorems and analysis, we now give a generalized dogleg algorithm for solving (18)-(19).
Then, we give the predicted decent bound in each iteration, which is the lower bound of the predicted reduction in each iteration: where ψ () is defined by (18).
Theorem 6. Suppose that (5) where This theorem is similar to that in [9] and its proof is omitted.

New Quasi-Newton Algorithm and Its Global Convergence
In this section, we propose a quasi-Newton method with a fractional model for linearly equality constrained optimization and prove its convergence under some reasonable conditions.In order to solve problem (1)-( 2), we consider the fractional model approximation for () about  = 0; that is where  =   + =   + 2 ,   = (  ), and ψ () is defined as (18).Thus,   (0) =   , g , and B are the corresponding gradient and Hessian approximations of the function at the th iteration.We choose   to minimize   ().There is a unique minimizer if and only if B is positive definite.In the following, we give our algorithm.If the current iteration is the feasible point   , then an equivalent form of ( 1)-( 2) is to solve the reduced unconstrained problem min In the following, we consider the choice of the parameter vectors ã , b , and c .We choose these vectors such that (53) satisfies the following conditions: where Obviously, (55) holds.Then, from (56), we have where and If we choose then these unknown parameters  1 ,  2 , and  3 can be obtained from (58)-(59).In the following, we give the derivation process of  1 ,  2 , and  3 .First, we define some notations: where the vectors  1 and  2 are chosen to satisfy α = γ = ζ = γ = 0.
For convenience, we omit the index  of   , α , α ,   , . . ., γ and γ .On one hand, from (58) we have where If the sequence { f } is monotonically decreasing and B is positive definite, then we know that  > 0 and (64) becomes On the other hand, by left-multiplying   1 on (59) and combining with (63), we have Then, from (66), we have where Similarly, by left multiplying   2 on (59), from ( 63) and (66) we have where ι =  − .Substituting (68) into the above equation, we have where And then from (66), we have Now we give the new quasi-Newton algorithm based on the fractional model (53).
Next we present the global convergence theorem which says the reduced gradients converge to zero.Theorem 8. Assume that (5) hold for all .From (52) and the assumptions in the theorem, we have where  1 and  2 are some positive constants.Then, from Step 10 of Algorithm 7, we have Since () is bounded from below and  +1 <   for all , we have that ∑ ∞ =1 min{ Δ ,  2 } is convergent, and Δ → 0 as  → ∞.

Numerical Tests
In this section, Algorithm 7 (abbreviated as FTR) is tested with some test problems which are chosen from [16,17].These test problems are listed in Table 1.We choose linear constrained problems HS9, HS48, HS49, HS50, Chen 3.3.1,and Chen 3.3.2.Moreover, in order to test Algorithm 7 more generally, we designed some problems where the objective functions are Pro.7-18 (see [14,18]) and the linear equality constraints are Pro.1-6.If b = c = 0 in Algorithm 7, we can obtain the conic model algorithm and call this algorithm CTR.We solve the following 18 test problems by FTR and CTR and compare their results.All the computations are carried out in Matlab R2012b on a microcomputer in double precision arithmetic.These tests use the same stopping criterion ‖g  ‖ ⩽ 10 −4 .The columns in the tables have the following meanings: Pro.denotes the numbers of the test problems;  is the dimension of the test problems; Iter is the number of iterations; nf and ng are the numbers of function and gradient evaluations, respectively; ‖g‖ is the Euclidean norm of the final reduced gradient; CPU(s) denotes the total iteration time of the algorithm in seconds.The parameters in these algorithms are The numerical comparison for 18 small-scale test problems is listed in Table 2.We can see that FTR is better than CTR for 15 tests in the number of iterations and the remaining 3 tests are similar.Because FTR needs some extra algebra computation for some parameters, FTR takes more time than CTR for small problems.
The numerical results of some large-scale problems are presented in Table 3. From Table 3, we find that for large-scale problems the CPU time of FTR is approximately the same as that of CTR but it has fewer number of iterations.From the above comparison, we see that FTR is slightly more effective and robust for these large-scale test problems.
The fractional model in Algorithm 7 is the extension of conic model.By using more information of function and gradient from the previous iterations and choosing parameters flexibly, the fractional model can be more approximate to the original problem.And the global convergence of the proposed quasi-Newton trust region algorithm is also proved.Numerical experiment shows the algorithm is effective and robust, including for large-scale test problems.The theoretical results and the numerical results lead us to believe that the method is worthy of further study.For example, we can consider using fractional model to solve the nonlinear equality constrained optimization problem.
and (35) hold, where  1 ∈ (0, 1/3).If  is continuously differentiable and bounded below in some set containing all iterations generated by Algorithm 7, the sequences {‖g  ‖} and {‖ B ‖} are uniformly bounded.Then, Proof.Assume that the theorem is false and there is  > 0 such that g ⩾  for all .From the assumption, we can assume that ⩽ B

Table 2 :
The numerical results of Algorithm 7 for some test problems.

Table 3 :
The numerical results of Algorithm 7 for some test problems ( = 3000).