A Modified Gradient Based Algorithm for Solving Matrix Equations

In this paper, we develop a modified gradient based algorithm for solving matrix equations AXB + CXD = F. Different from the gradient based method introduced by Xie et al., 2010, the information generated in the first half-iterative step is fully exploited and used to construct the approximate solution.Theoretical analysis shows that the new method converges under certain assumptions. Numerical results are given to verify the efficiency of the new method.


Introduction
Consider a linear matrix equation of the following form: where  ∈ R × ,  ∈ R × ,  ∈ R × ,  ∈ R × , and  ∈ R × are the given constant matrices and  ∈ R × is the unknown matrix to be solved.A Sylvester equation  +    =  is the special case of (1) with  =  =  and  =  =   , the notation   is the identity matrix of  × .Such kind of problems frequently arise from many areas of applications in control and system theory [1], stability of linear systems [2], analysis of bilinear systems [3], power systems [4], signal and image processing [5], and so forth.The exact solutions of matrix equations, such as Lyapunov and Sylvester matrix equations, can be obtained from matrix inversion by using the Kronecker product.The drawback of this approach is that considerable computational costs and storage requirements are needed, so that this approach is only applicable for small sized Sylvester equations.Some direct methods have also been proposed in [6][7][8][9] which are based on the idea of transforming the coefficient matrix into a Schur or Hessenberg form, by which the original equation can be solved by a backward substitution.
In the numerical linear community, iterative methods are becoming more and more popular.Several iterative schemes for Sylvester equations have been proposed; see, for example, [10][11][12][13][14][15].Recently, some efficient gradient based and least squares based iterative algorithms for solving generalized Sylvester equations and coupled (general coupled) Sylvester equations have been presented [16][17][18][19][20][21][22][23][24][25][26][27][28].The basic idea of these approaches is based on a hierarchical identification principle [16][17][18], which regards the unknown matrix as the system parameter matrix to be identified and then constructs a recursive formula to approximate the unknown solution.Particularly, for general linear matrix equations of form (1), it is illustrated in [3,4] that the unknown matrix to be identified can be computed by a gradient based iterative algorithm.The convergence properties of the methods are also investigated in [3].In this paper, a modified gradient based iterative algorithm is proposed for solving linear matrix equations of form (1). The information generated in the first half-iterative step is fully exploited and used to construct the approximate solution by the modified method.The convergence condition of the method is analyzed.The numerical performance of the method is compared with the algorithms in [3,4].Numerical results show that the new method is efficient and robust.
The paper is organized as follows.In Section 2, the gradient based iterative method is recalled, and the modified gradient based method is introduced and analyzed in Section 3. In Section 4, performance of the modified gradient based method is compared with the original one.Finally, we conclude the paper in Section 5.

A Brief Review of the Gradient Based Iterative Method
We firstly recall an iterative method proposed by Xie et al. [3] for solving (1).The basic idea is regarding (1) as two linear matrix equations as follows: Then, define two recursive sequences as follows: where  is the iterative step size.The above procedures can be regarded as two separate iterative procedures for solving two matrix equations in (3).
With  (1)   and  (2)   at hand, then the th approximate solution   can be defined by taking the average of two approximate solutions, that is, By selecting an appropriate initial approximate solution  0 , and using  −1 to substitute  (1)  −1 in (4) and  (2)  −1 in ( 5), then the above ( 4)-( 6) constitute the gradient based iterative method proposed in [3].It is shown in [3] that the gradient based iterative algorithm converges as long as where  max (  ) is the largest eigenvalue of   .According to lots of numerical experiments, GBI algorithm is computationally efficient.However, we observe that the GBI algorithm has some limitations.The convergent rate is slow and the stagnation will happen for ill-conditioned problem.Also, in [3], the authors pointed out that how to choose a best convergence factor is a subject to be studied and deserves further research.In this paper, we present the optimal convergence factor explicitly, and then propose a modified algorithm for solving the linear matrix equations (1).

A Modified Gradient Based Iterative Algorithm
The above GBI process can be accomplished by the following algorithm.
The following result discusses the convergence conditions of the Algorithm 2.

Theorem 4. If the linear matrix equation (1) has a unique solution 𝑋 and
Journal of Applied Mathematics 3 then the iterative sequence   generated by Algorithm 2 converges to ; that is, lim  → ∞   = ; or the error   =   −  converges to zero for any initial value  0 .
Proof.In the following discussions, we always assume that the Frobenius matrix norm is used.For the clarity of proof, we introduce another variable X−1 instead of  −1 in the fifth iteration.Define the following error matrices: =  (1)   − , By using ( 1) and ( 10), the proof of the following equalities is trivial: Taking the Frobenius norm of both sides of (11), it follows that       (1) From   = [ (1)   +  (2)   ]/2, we have Obviously, ∑  =1 (‖ Ê− ‖ 2 /2  ) < ∞.In fact, the iterative sequence X ,  = 0, 1, . . .generated by Algorithm 2 can also be viewed as the sequence generated by the doubleside iteration in [3], so lim It follows that or According to Lemma 3, we have  −1 → 0 as  → ∞.From (8), the exact solution is
Remark.The choice of the convergence factor  is an important issue.We experimentally study its influence on the convergence.The effect of changing of the convergence factor  for MGBI algorithms in Example ?? is illustrated in Figure 2. We see that is a better convergence factor.However, the convergence factor is problem dependent, so seeking a best convergence factor is still a difficult task.Example 6. Suppose that  +    = , where ) .(20) Then, the solution of  from ( 8) is ) .
The coefficient matrices used in this example are taken from [4].Taking  0 = 10 −6  2 , we apply the GBI algorithm and MGBI algorithm to compute   , the convergence factor  is set to be 2/[ max (  )+ max (  )] = 0.27 in GBI algorithm and to be min{2/ max (  ), 2/ max (  )} = 0.377 in MGBI algorithm.The relative error  := ‖  − ‖/‖‖ is recorded in Figure 3. From the figure, we can also see that the MGBI algorithm converges faster than the GBI algorithm.
The effect of changing of the convergence factor  for MGBI algorithms in Example 6 is illustrated in Figure 4.

Conclusions
In this paper, a modified gradient based iteration (MGBI) method is proposed for linear matrix equation.The convergence of MGBI is analyzed.The choice of parameter  is an important issue, and its influence is experimentally studied.The principle idea of this paper can be extended to the more general setting like generalized (coupled) Sylvester matrix equations.