Journal of Applied Mathematics
Volume 2015 (2015), Article ID 562529, 12 pages
http://dx.doi.org/10.1155/2015/562529

Research Article

LSMR Iterative Method for General Coupled Matrix Equations

F. Toutounian,1,2 D. Khojasteh Salkuyeh,³ and M. Mojarrab¹

¹Department of Applied Mathematics, School of Mathematical Sciences, Ferdowsi University of Mashhad, Iran
²The Center of Excellence on Modelling and Control Systems, Ferdowsi University of Mashhad, Iran
³Faculty of Mathematical Sciences, University of Guilan, Iran

Received 18 March 2014; Accepted 28 July 2014

Academic Editor: D. R. Sahu

Copyright © 2015 F. Toutounian et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

By extending the idea of LSMR method, we present an iterative method to solve the general coupled matrix equations , , (including the generalized (coupled) Lyapunov and Sylvester matrix equations as special cases) over some constrained matrix groups , such as symmetric, generalized bisymmetric, and -symmetric matrix groups. By this iterative method, for any initial matrix group , a solution group can be obtained within finite iteration steps in absence of round-off errors, and the minimum Frobenius norm solution or the minimum Frobenius norm least-squares solution group can be derived when an appropriate initial iterative matrix group is chosen. In addition, the optimal approximation solution group to a given matrix group in the Frobenius norm can be obtained by finding the least Frobenius norm solution group of new general coupled matrix equations. Finally, numerical examples are given to illustrate the effectiveness of the presented method.

1. Introduction

In control and system theory [1–7], we often encounter Lyapunov and Sylvester matrix equations which have been playing a fundamental role. Due to the important roles of the matrix equations, the studies on the matrix equations have been addressed in a large body of papers [8–27]. By using the hierarchical identification principle [9–11, 28–32], a gradient-based iterative (GI) method was presented to solve the solutions and the least-squares solutions of the general coupled matrix equations. In [19, 33], Zhou et al. deduced the optimal parameter of the GI method for solving the solutions and the weighted least-squares solutions of the general coupled matrix equations. Dehghan and Hajarian [34–36] introduced several iterative methods to solve various linear matrix equations.

In [12, 17], Huang et al. presented finite iterative algorithms for solving generalized coupled Sylvester systems. Li and Huang [37] proposed a matrix LSQR iterative method to solve the constrained solutions of the generalized coupled Sylvester matrix equations. Hajarian [38] presented the generalized QMRCGSTAB algorithm for solving Sylvester-transpose matrix equations. Recently, Lin and Simoncini [39] established minimal residual methods for large scale Lyapunov equations. They explored the numerical solution of this class of linear matrix equations when a minimal residual (MR) condition is used during the projection step.

In this paper, we construct a matrix iterative method based on the LSMR algorithm [40] to solve the constrained solutions of the following problems

Compatible matrix equations are as follows:

Least-squares problem is as follows:

Matrix nearness problem is as follows:

where , , , , , are constant matrices with suitable dimensions, , , are unknown matrices to be solved, , , are given matrices, and is the solution set of (1) or problem (2).

This paper is organized as follows. In Section 2, we will briefly review the LSMR algorithm for solving linear systems of equations. In Section 3, we propose the matrix LSMR iterative algorithms for solving the problems (1)-(2). In Section 4, we solve the problem (3) by finding the minimum Frobenius norm solution group of the corresponding new general coupled matrix equations. In Section 5, numerical examples are given to illustrate the efficiency of the proposed iterative method. Finally, we make some concluding remarks in Section 6.

The notations used in this paper can be summarized as follows. represents the trace of the matrix . For , notation is Kronecker product and is the inner product with the Frobenius norm . The use of represents the vector operator defined as

where is the th column of . The generalized bisymmetric matrices, the -symmetric matrices, and the symmetric orthogonal matrices can be defined as follows.

Definition 1 (see [41]). A matrix is said to be a symmetric orthogonal matrix (), if and .

Definition 2 (see [42]). For given symmetric orthogonal matrices , , we say a matrix is -symmetric (), if .

Definition 3 (see [43]). For a given symmetric orthogonal matrix , a matrix is said to be a generalized bisymmetric matrix (), if and .

2. LSMR Algorithm

In this section, we briefly review some fundamental properties of the LSMR algorithm [40], which is an iterative method for computing a solution to either of the following problems.

Compatible linear systems are as follows:

Least-squares problem is as follows:

where and . The LSMR algorithm uses an algorithm of Golub and Kahan [44], which stated as procedure Bidiag 4, to reduce to lower bidiagonal form. The procedure Bidiag 4 can be described as follows.

Bidiag 4 (starting vector ; reduction to lower bidiagonal form).

The scalers and are chosen such that .
The following properties presented in [7] illustrate that the procedure Bidiag 4 has finite termination property.

Property 1. Suppose that steps of the procedure Bidiag 4 have been taken; then the vectors and are the orthonormal basis of the Krylov subspaces and , respectively.

Property 2. The procedure Bidiag 4 will stop at step if and only if is , where is the grade of with respect to and is the grade of with respect to .

By using the procedure Bidiag 4, the LSMR method constructs an approximation solution of the form , where , which solves the least-squares problem, , where is the residual for the approximate solution . The main steps of the LSMR algorithm can be summarized as shown in Algorithm 1.

Set , , , , , , , , , ,

For , until convergence Do:

      If is small enough then stop

End Do.

Algorithm 1: LSMR algorithm.

More details about the LSMR algorithm can be found in [40].

The stopping criterion may be used as for the compatible linear systems (5) and for the least-squares problem (6). Other stopping criteria can also be used and are not listed here. Reader can see [40] for details. Clearly, the sequence generated by the LSMR algorithm converges to the unique minimum norm solution of (5) or the unique minimum norm least-squares solution of problem (6).

3. A Matrix LSMR Iterative Method

In this section, we will present our matrix iterative method based on the LSMR algorithm, for solving (1) and problem (2). For the unknown matrices , , by using the Kronecker product, (1) and problem (2) are equivalent to (5) and problem (6), respectively, with

Hence, by using the invariance of the Frobenius norm under unitary transformations, it is easy to prove that the vector form , , , and () in LSMR algorithm can be rewritten in matrix forms, respectively, as

If the unknown matrices , then (1) and problem (2) are equivalent to (5) and problem (6), respectively, with

where . Hence, the vector forms of , , , and () in LSMR algorithm can be rewritten in matrix forms, respectively, as

If the unknown matrices , then (1) and problem (2) are equivalent to (5) and problem (6), respectively, with

where , . Hence, the vector forms of , ,  , and () in LSMR algorithm can be rewritten in matrix forms, respectively, as

If the unknown matrices (the set of symmetric matrices), then (1) and problem (2) are equivalent to (5) and problem (6), respectively, with

Hence, the vector forms of , , , and () in LSMR algorithm can be rewritten in matrix forms, respectively, as

From above results, we can obtain the matrix form iteration method of LSMR algorithm for solving the constrained solution group of (1) and problem (2). When the unknown matrices , the matrix form iterative method is given as shown in Algorithm 2.

Set ,    

      , ,

      ,

Set , , , , , , ,

For , until converges Do:

      ,

      ,

      ,

      ,

      ,

      If is small enough then stop

End Do.

Algorithm 2: LSMR_SR_M algorithm.

4. The Solution Group of Problem (3)

Now, we consider the solution group of the matrix nearness problem (3) for given matrix group , where ,  . If , it is easy to prove that

Let

then problem (3) is equivalent to finding the minimum Frobenius norm symmetric solution group or minimum Frobenius norm least-squares symmetric solution group of the following problems, respectively.

Compatible matrix equations are as follows:

Least-squares problem is as follows:

By LSMR_SR_M method, we can get the minimum Frobenius norm symmetric solution group of (19) (or the minimum Frobenius norm least-squares symmetric solution group of problem (20)). Then, the optimal approximate solution group of problem (3) can be obtained; that is, .

5. Numerical Examples

To compare the behavior of the proposed matrix method discussed in the previous section with the CGNE method [43] and the matrix LSQR iterative method (LSQR_M) [37], we present in this section numerical results for three examples. All the numerical computations are performed in MATLAB 7.

Example 1. Suppose that the matrices ,  ,  ,  , are given by the following matrices:

The and matrices are chosen such that and , where and are the identity matrix and the matrix whose entries are all one, respectively.

In Figure 1, we display the convergence curves of the function , with

where ,  , is the residual matrix of the th equation in th iteration. The initial iterative matrices in all the iterative methods are chosen as zero matrices of suitable size. Figure 1 confirms that the proposed algorithm has faster convergence rate and higher accuracy than the CGNE method and similar behavior to the matrix LSQR iterative method.

Figure 1: Convergence history of the LSMR, CGNE, and LSQR iterative methods for Example 1.

Example 2. Suppose that the matrices ,  ,  ,  , are given by the following matrices:

As Example 1, the and matrices are chosen such that and with and the initial iterative matrices in all the iterative methods are chosen as zero matrices of suitable size. In Figure 2, as Figure 1, we display the convergence curves of the function ,. This figure shows that the LSMR method outperforms the CGNE and LSQR methods.

Figure 2: Convergence history of the LSMR, CGNE, and LSQR iterative methods for Example 2.

Example 3 (see [45]). Consider the convection diffusion equation with the Dirichlet boundary conditions

Here is the unit square . The operator was discretized using central finite differences on , with mesh size in the “” direction and in the “” direction. This yields a linear system of algebraic equations that can be written as a Sylvester matrix equation

(as a particular case of (1) with ,  ,  ,  ,  , and ) where tridiagonal matrices and are given by

The right-hand side matrix is obtained as follows:

In this example, the functions and were chosen such that the exact solution is

on the domain . In addition, we used the symmetric successive overrelaxation (SSOR) preconditioner for the matrix equation (25) to increase the convergence rate. It is easy to prove that the matrix equation (25) is equivalent to the linear system:

where ,  , and .
The matrices and can be written as

where is the diagonal of and and are the strict lower and upper part of , respectively. Then the splitting of the matrix is given as

with

Now instead of solving the matrix equation (25), we will apply the LSMR-M algorithm to the preconditioned system

where is a preconditioner. As said, we use the SSOR preconditioner defined by

We note that the matrix is not used explicitly. We only use the action of the linear operator on a matrix , defined by . In addition, we use only matrix-by-vector products; then when using the SSOR preconditioner we have to compute, for a given , the matrix such that

or

With setting

the linear system (35) is equivalent to

For computing such that , we have to solve the following matrix equations:

The matrix equations (39) and (41) are also Sylvester matrix equations. But as was stated in [45], since the matrices involved in these equations are triangular, they are solved easily. In (39), the matrix can be computed from left to right and from top to bottom in each column; this corresponds to backward substitution. Equation (41) is solved in the opposite sense and this corresponds to forward substitution. Now, to compute in (35), it is sufficient to use the action of the operator on the matrix , defined by .

To compute in (36), first, we use the action of the operator on the matrix, defined by . Then, by setting

the linear system (36) is equivalent to

Therefore, can be obtained by solving the following matrix equations:

Similarly, the matrix equations (44) and (46) are also Sylvester matrix equations. But since the matrices involved in these equations are triangular, in (44), the matrix can be computed from right to left and from bottom to top in each row; this corresponds to forward substitution. Equation (46) is solved in the opposite sense and this corresponds to backward substitution.

In Figure 3, we exhibited the function with

versus the number of iterations for LSMR-M and the SSOR-LSMR-M. Furthermore, we note that for computing the quantity , ( is the residual matrix in th iteration) we used the pseudocode stated in [40]. These results were obtained for ,  ,  , and . The initial iterative matrix was chosen as zero matrix of suitable size. As we observe by using the SSOR preconditioner the convergence rate of the LSMR-M algorithm has increased, effectively.

Figure 3: Convergence history of the LSMR-M and SSOR-LSMR-M.

6. Conclusion

Solving the linear matrix equations is an attractive part of research. By extending the idea of LSMR method, we have proposed Algorithm 2 to solve the coupled matrix equations (1) or the least-squares problem (2) over generalized symmetric matrices. By this new iterative method on the selection of special initial matrix group, we obtain the minimum Frobenius norm solutions or the minimum Frobenius norm least-squares solutions over generalized symmetric matrices. All the presented results show that the matrix LSMR iterative method is efficient to compute the solution group of the general coupled matrix equations.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgment

The authors would like to thank the referees for their valuable remarks and helpful suggestions.

References

T. Chen and B. A. Francis, Optimal Sampled-Data Control Systems, Springer, London, UK, 1995.

T. Chen and L. Qiu, “ design of general multirate sampled-data control systems,” Automatica, vol. 30, no. 7, pp. 1139–1152, 1994.

L. P. LaSalle and S. Lefschetz, Stability by Lyapunov's Direct Method, Academic Press, New York, NY, USA, 1961.

Z. Y. Li, Y. Wang, B. Zhou, and G. R. Duan, “Least squares solution with the minimum-norm to general matrix equations via iteration,” Applied Mathematics and Computation, vol. 215, no. 10, pp. 3547–3562, 2010.

B. C. Moore, “Principal component analysis in linear systems: controllability, observability, and model reduction,” IEEE Transactions on Automatic Control, vol. 26, no. 1, pp. 17–32, 1981.

L. Qiu and T. Chen, “Multirate sampled-data systems: all suboptimal controllers and the minimum entropy controller,” IEEE Transactions on Automatic Control, vol. 44, no. 3, pp. 537–550, 1999.

F. Toutounian and S. Karimi, “Global least squares method (Gl-LSQR) for solving general linear systems with several right-hand sides,” Applied Mathematics and Computation, vol. 178, no. 2, pp. 452–460, 2006.

M. Dehghan and M. Hajarian, “An efficient algorithm for solving general coupled matrix equations and its application,” Mathematical and Computer Modelling, vol. 51, no. 9-10, pp. 1118–1134, 2010.

F. Ding and T. Chen, “Gradient based iterative algorithms for solving a class of matrix equations,” IEEE Transactions on Automatic Control, vol. 50, no. 8, pp. 1216–1221, 2005.

F. Ding and T. Chen, “Iterative least-squares solutions of coupled Sylvester matrix equations,” Systems & Control Letters, vol. 54, no. 2, pp. 95–107, 2005.

F. Ding and T. Chen, “On iterative solutions of general coupled matrix equations,” SIAM Journal on Control and Optimization, vol. 44, no. 6, pp. 2269–2284, 2006.

G. X. Huang, N. Wu, F. Yin, Z. L. Zhou, and K. Guo, “Finite iterative algorithms for solving generalized coupled SYLvester systems—part I: one-sided and generalized coupled SYLvester matrix equations over generalized reflexive solutions,” Applied Mathematical Modelling, vol. 36, no. 4, pp. 1589–1603, 2012.

I. Jonsson and B. Kågström, “Recursive blocked algorithm for solving triangular systems. I. One-sided and coupled Sylvester-type matrix equations,” Association for Computing Machinery. Transactions on Mathematical Software, vol. 28, no. 4, pp. 392–415, 2002.

I. Jonsson and B. Kågström, “Recursive blocked algorithms for solving triangular systems-Part II: two-sided and generalized Sylvester and Lyapunov matrix eqations,” ACM Transactions on Mathematical Software, vol. 28, no. 4, pp. 416–435, 2002.

Z. H. Peng, X. Y. Hu, and L. Zhang, “The bisymmetric solutions of the matrix equation image and its optimal approximation,” Linear Algebra and Its Applications, vol. 426, no. 2-3, pp. 583–595, 2007.

G. Starke and W. Niethammer, “SOR for ,” Linear Algebra and Its Applications, vol. 154–156, no. 3, pp. 355–375, 1991.

C. Tsui, “New approach to robust observer design,” International Journal of Control, vol. 47, no. 3, pp. 745–751, 1988.

F. Yin, G. Huang, and D. Chen, “Finite iterative algorithms for solving generalized coupled SYLvester systems, Part II: two-sided and generalized coupled SYLvester matrix equations over reflexive solutions,” Applied Mathematical Modelling, vol. 36, no. 4, pp. 1604–1614, 2012.

B. Zhou, G. Duan, and Z. Li, “Gradient based iterative algorithm for solving coupled matrix equations,” Systems & Control Letters, vol. 58, no. 5, pp. 327–333, 2009.

Z. Chen and L. Lu, “A gradient based iterative solutions for Sylvester tensor equations,” Mathematical Problems in Engineering, vol. 2013, Article ID 819479, 7 pages, 2013.

D. Chen, F. Yin, and G. X. Huang, “An iterative algorithm for the generalized reflexive solution of the matrix equations , ,” Journal of Applied Mathematics, vol. 2012, Article ID 492951, 20 pages, 2012.

F. Ding, “Combined state and least squares parameter estimation algorithms for dynamic systems,” Applied Mathematical Modelling. Simulation and Computation for Engineering and Environmental Systems, vol. 38, no. 1, pp. 403–412, 2014.

F. Ding, Y. Liu, and B. Bao, “Gradient-based and least-squares-based iterative estimation algorithms for multi-input multi-output systems,” Proceedings of the Institution of Mechanical Engineers. Part I. Journal of Systems and Control Engineering, vol. 226, no. 1, pp. 43–55, 2012.

K. Wang, Z. Liu, and C. Xu, “A modified gradient based algorithm for solving matrix equations ,” Journal of Applied Mathematics, vol. 2014, Article ID 954523, 6 pages, 2014.

H. Yin and H. Zhang, “Least squares based iterative algorithm for the coupled sylvester matrix equations,” Mathematical Problems in Engineering, vol. 2014, Article ID 831321, 8 pages, 2014.

H. Zhang and F. Ding, “A property of the eigenvalues of the symmetric positive definite matrix and the iterative algorithm for coupled Sylvester matrix equations,” Journal of the Franklin Institute, vol. 351, no. 1, pp. 340–357, 2014.

J. Zhou, R. Wang, and Q. Niu, “A preconditioned iteration method for solving Sylvester equations,” Journal of Applied Mathematics, vol. 2012, Article ID 401059, 12 pages, 2012.

F. Ding and T. Chen, “Hierarchical gradient-based identification of multivariable discrete-time systems,” Automatica, vol. 41, no. 2, pp. 315–325, 2005.

F. Ding and T. Chen, “Hierarchical least squares identification methods for multivariable systems,” IEEE Transactions on Automatic Control, vol. 50, no. 3, pp. 397–402, 2005.

F. Ding, X. Liu, H. Chen, and G. Yao, “Hierarchical gradient based and hierarchical least squares based iterative parameter identification for CARARMA systems,” Signal Processing, vol. 97, pp. 31–39, 2014.

F. Ding, P. X. Liu, and J. Ding, “Iterative solutions of the generalized Sylvester matrix equations by using the hierarchical identification principle,” Applied Mathematics and Computation, vol. 197, no. 1, pp. 41–50, 2008.

Y. Liu, F. Ding, and Y. Shi, “An efficient hierarchical identification method for general dual-rate sampled-data systems,” Automatica, vol. 50, no. 3, pp. 962–973, 2014.

B. Zhou and G. R. Duan, “On the generalized Sylvester mapping and matrix equations,” Systems & Control Letters, vol. 57, no. 3, pp. 200–208, 2008.

M. Dehghan and M. Hajarian, “Efficient iterative method for solving the second-order Sylvester matrix equation ,” IET Control Theory & Applications, vol. 3, no. 10, pp. 1401–1408, 2009.

M. Dehghan and M. Hajarian, “The general coupled matrix equations over generalized bisymmetric matrices,” Linear Algebra and its Applications, vol. 432, no. 6, pp. 1531–1552, 2010.

M. Hajarian and M. Dehghan, “The generalized centro-symmetric and least squares generalized centro-symmetric solutions of the matrix equation ,” Mathematical Methods in the Applied Sciences, vol. 34, no. 13, pp. 1562–1579, 2011.

S. Li and T. Huang, “LSQR iterative method for generalized coupled Sylvester matrix equations,” Applied Mathematical Modelling, vol. 36, no. 8, pp. 3545–3554, 2012.

M. Hajarian, “The generalized QMRCGSTAB algorithm for solving Sylvester-transpose matrix equations,” Applied Mathematics Letters, vol. 26, no. 10, pp. 1013–1017, 2013.

Y. Lin and V. Simoncini, “Minimal residual methods for large scale Lyapunov equations,” Applied Numerical Mathematics, vol. 72, pp. 52–71, 2013.

D. C. Fong and M. Saunders, “LSMR: an iterative algorithm for sparse least-squares problems,” SIAM Journal on Scientific Computing, vol. 33, no. 5, pp. 2950–2971, 2011.

M. Dehghan and M. Hajarian, “An iterative algorithm for the reflexive solutions of the generalized coupled Sylvester matrix equations and its optimal approximation,” Applied Mathematics and Computation, vol. 202, no. 2, pp. 571–588, 2008.

M. L. Liang, L. F. Dai, and S. F. Wang, “An iterative method for -symmetric solution of matrix equation ,” Scientia Magna, vol. 4, no. 3, pp. 60–70, 2008.

M. Dehghan and M. Hajarian, “An iterative method for solving the generalized coupled Sylvester matrix equations over generalized bisymmetric matrices,” Applied Mathematical Modelling, vol. 34, no. 3, pp. 639–654, 2010.

G. Golub and W. Kahan, “Calculating the singular values and pseudo-inverse of a matrix,” Journal of the Society for Industrial and Applied Mathematics B: Numerical Analysis, vol. 2, no. 2, pp. 205–224, 1965.

A. Bouhamidi and K. Jbilou, “A note on the numerical approximate solutions for generalized Sylvester matrix equations with applications,” Applied Mathematics and Computation, vol. 206, no. 2, pp. 687–694, 2008.