Computing Weighted Analytic Center for Linear Matrix Inequalities Using Infeasible Newton’s Method

. We study the problem of computing weighted analytic center for system of linear matrix inequality constraints. The problem can be solved using Standard Newton’s method. However, this approach requires that a starting point in the interior point of the feasible region be given or a Phase I problem be solved. We address the problem by using Infeasible Newton’s method applied to the KKT system of equations which can be started from any point. We implement the method using backtracking line search technique and also study the effect of large weights on the method. We use numerical experiments to compare Infeasible Newton’s method with Standard Newton’s method. The results show that Infeasible Newton’s method moves in the interior of the feasible regions often very quickly, starting from any point. We recommend it as a method for finding an interior point by setting each weight to be 1. It appears to work better than Standard Newton’s method in finding the weighted analytic center when none of weights is very large relative to the other weights. However, we find that Infeasible Newton’s method is more sensitive than Standard Newton’s method to large variation in the weights.


Introduction
We consider a system of linear matrix inequality constraints given as follows: subject to  () () := where  ∈ R  is a variable and each  ()  is an   ×   symmetric matrix.Linear matrix inequality (LMI) constraints have been well studied especially in the field of semidefinite programming [1,2].LMI constraints have applications in a variety of fields including engineering, geometry, and statistics.We assume that feasibility determined by the constraints is bounded and has a nonempty interior.This means that the set {diag( (1)  1 , . . .,  () 1 ), . . ., diag( (1)   , . . .,  ()  )} is linearly independent [3].
In this paper, we are concerned with computing weighted analytic center for LMIs using Infeasible Newton's method.
A feasible starting point is not required to start the method.In the special case of linear constraints, weighted analytic center has been studied extensively in the past (e.g., [4]).A weighted analytic center for LMIs which extends the definition given in [4] was given in [5,6].The study weighted analytic center is of interest in its own right.Many algorithms for linear programming and semidefinite programming are based on weighted analytic centers [2,3,7,8].
Weighted analytic center for linear matrix inequalities can be found using Standard Newton's method by minimizing the barrier function.This approach has the drawback that a starting in the interior of the feasible region must be given.Also, Newton's method does not work well when some of the weights are relatively very large relative to the other weights.Infeasible Newton's method for analytic center for single LMI constraint is given in [9].We present Infeasible Newton's method for finding weighted analytic center that can be started from any point.The method is applied to the Karush-Kuhn-Tucker (KKT) system of equations for the weighted analytic center problem.We implement the method using backtracking line search technique and also study the effect of large weights on the method.We use numerical experiments to compare Infeasible Newton's method with Standard Newton's method.
We find that Infeasible Newton's method moves very quickly into the interior of the feasible regions for most of our test problems.It seems to be a suitable method for finding an interior point for the system by setting each weight to be 1.It works better than Standard Newton's method if none of the weights is relatively very large with respect to the other weights.We also find that Infeasible Newton's method is more sensitive to large variations in the weights than Standard Newton's method.In the case of very large variation in the weights, we recommend using Infeasible Newton's method to get into the interior with each weight set to 1 and then switching to Standard Newton's method for convergence to the weighted analytic center using the original weights and starting from the interior.

Weighted Analytic Center for Linear Matrix Inequalities
Let R denote the feasible region defined by inequalities (1).
Given  > 0, define the barrier function   () : R  → R by The weighted analytic center for system (1) is defined by [5,6]  ac () = arg min This definition extends that given in [4] for linear constraints.When  = [1, . . ., 1],  ac () is called the analytic center.Weighted analytic center has been used in interior point methods for linear programs and semidefinite programs [2,3,7,8,10].The primal-dual central path in semidefinite programming converges to the analytic center of the optimal solution set [11].
Standard Newton's method has the choice for finding weighted analytic center.The gradient and Hessian of the barrier function   () are given by [6] the following: for ,  = 1, . . .,

Standard Newton's Method for Computing Weighted Analytic Center
Input: An interior point , tolerance TOL > 0 Set  = 1

Repeat
(1) Compute the Newton's direction  = −[()] −1 ∇  () Line search technique such as backtracking line search technique can be used in Newton's method to find weighted analytic center [9].

Infeasible Newton's Method for Computing Weighted Analytic Center
In this section, we describe Infeasible Newton's method for finding weighted analytic center.The problem of computing the weighted analytic center in (3) is a more general form of the determinant maximization problem [3].Its dual is given by the following: where the bullet • is the matrix dot-product.Theorem 1 gives optimality conditions for computing the weighted analytic center  ac ().
Step 4. Do line search to get stepsize ℎ.

Numerical Experiments
In this section, we give numerical experiments to compare Infeasible Newton's method with Standard Newton's method.We also investigate the effects of large weights on the two methods.Table 1 describes the 35 random test problems used for our numerical experiments.The second column of Table 1 gives the dimension  of the ambient space and the third column is the number  of constraints.The dimensions   of the matrices are given in the fourth column.For each random problem, , , and   are given and the LMI ⪰ 0 was generated randomly as follows:  () 0 is an   ×   diagonal matrix with each diagonal entry chosen from (0, 1).Each  Our codes were written in MATLAB and ran on Dell OPTIPLEX 880 computer.Both Infeasible Newton's method and Standard Newton's method were implemented using a tolerance of TOL = 10 −4 and up to a maximum of 500 iterations.The starting point is random such that each of its components is chosen from a normal distribution with mean 0 and variance 10 6 .We use the backtracking line search technique in the two methods.Table 2 compares Infeasible Newton's method with Standard Newton's method for different sets of weights.In each of Problems 1-15, one weight was set at 10 12 , which is a very high value relative to the others.For Problems 16-35, none of the weights was relatively very large.In Table 2, the third and the fourth columns give the number of iterations and time required to find the first point in the interior of the feasible region (resp.)by Infeasible Newton's method.The fourth and the fifth columns give the number of iterations and time required to find the weighted analytic center (resp.)starting from the first interior point found.The sixth and the seventh columns in Table 2 give the number of iterations and time required to find the weighted analytic center (resp.)by Standard Newton's method.Standard Newton's method is started from the same interior point as in Infeasible Newton's method.
Figure 1 shows the iterates taken by Infeasible Newton's method to converge in Problem with  = [10 12 , 1000].It is clear from the figure that Infeasible Newton's method slowed down considerably before converging to the weighted analytic center.On the other hand, as seen in Figure 2, the method converged quickly with  = [1,1].Figure 3 shows how the norm of the gradient varies with the number of iterations for the two values of the weights.In Table 2,   the entry * means that Infeasible Newton's method has failed to converge after the maximum number of 500 iterations in Problems 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, and 29.However, it managed to find an interior point of (1) in Problems 2, 4, 5, 7, 9, 10, 12, and 29.We see from the table that both methods might not work well when some of the weights are relatively larger than the other weights.The results also show that Standard Newton's method performs better than Infeasible Newton's method in this case.In Infeasible Newton's method, the Jacobian of the residual function becomes increasingly ill conditioned near the boundary of systems ( 8) and ( 9) due to matrices  and  as the variation among the weights increases.Observe that Infeasible Newton's method failed to converge in 8 problems, even though it found a point in the interior of the feasible region (1).Standard Newton's method converged (within the 500 iterations' limit) for each of the 8 problems except in Problem 29.Standard Newton's method converged after 536 iterations in Problem 29.
In Table 3, the entry * in Problem 29 indicates that Infeasible Newton's method has failed to converge to the analytic center after the maximum number of 500 iterations.However, it managed to find an interior point of (1) after 1 iteration.Problem 29 shows both Infeasible Newton's method and Standard Newton's method may fail to converge (within 500 iterations) even if the weights are all equal and an interior point is found.We see from Table 3 that Infeasible Newton's method took a fewer number of iterations in 12 out of 35 problems while Standard Newton's method took a fewer number of iterations in 9 out of 35 problems.Infeasible Newton's method took less time in 23 out of 35 problems and Standard Newton's method took less time in 11 out of 35 problems.It is interesting to note from the table that Infeasible Newton's method found an interior point of (1) within 1-3 iterations on most of the test problems.
The results from Tables 2 and 3 suggest that when none of the weights is relatively very large, Infeasible Newton's method is a better method than Standard Newton's method to find the weighted analytic center.When one weight is relatively very large, one could use Infeasible Newton's method with  = [1, 1, . . ., 1] to find an interior point and then switch to Standard Newton's method using the original weights and starting from the interior point found.

Figure 2 :Figure 3 :
Figure 2: Iterates (1 + 9 = 10) taken by Infeasible Newton's method to converge in Problem 3 with  = [1, 1].The graph on the right is the graph on the left zoomed and showing the last iterate * well inside the interior of the feasible region.

Table 2 :
Infeasible Newton's versus Standard Newton's methods using different weights .