Mathematical Problems in Engineering

Volume 2014 (2014), Article ID 831321, 8 pages

http://dx.doi.org/10.1155/2014/831321

Research Article

Least Squares Based Iterative Algorithm for the Coupled Sylvester Matrix Equations

Hongcai Yin¹ and Huamin Zhang²

¹School of Management Science and Engineering, Anhui University of Finance & Economics, Bengbu 233000, China

²Department of Mathematics & Physics, Bengbu College, Bengbu 233030, China

Received 27 April 2014; Accepted 26 May 2014; Published 18 June 2014

Academic Editor: Ion Zaballa

Copyright © 2014 Hongcai Yin and Huamin Zhang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

By analyzing the eigenvalues of the related matrices, the convergence analysis of the least squares based iteration is given for solving the coupled Sylvester equations and in this paper. The analysis shows that the optimal convergence factor of this iterative algorithm is 1. In addition, the proposed iterative algorithm can solve the generalized Sylvester equation . The analysis demonstrates that if the matrix equation has a unique solution then the least squares based iterative solution converges to the exact solution for any initial values. A numerical example illustrates the effectiveness of the proposed algorithm.

1. Introduction

Matrix equations arise in systems and control, such as Lyapunov matrix equation and Riccati equation. How to solve these matrix equations becomes a major field of the matrix computations [1–4]. Main points of this field are the decompositions and transformations of the matrices, eigenvalues and eigenvectors, and Krylov subspace [5–8]. Other points contain the algorithms and the convergence analysis. The algorithms provided a set of operation steps, and according to these steps it can find a solution of a matrix equation within finite steps in given error bounds [9–11]. The convergence analysis offered more details of an algorithm, and in general, these details indicated some new research areas [12–15].

The direct method and the indirect method are the two main approaches of solving the matrix equations [16–18]. With the development of the requirement of the computation and the matrix theory, the indirect method or iterative method becomes the main approach of solving the matrix equations [19, 20].

Iterative method is active in studying other engineering problems. Particularly, the iterative method can be used to identify systems [21–23] and estimate parameter of systems [24–28]. For example, iterative method is useful in the identification [29–32] and parameter estimation [33–36] of linear and nonlinear systems [37–46].

The Jacobi and Gauss-Seidel iterative methods were discussed in the literature. By extending the Jacobi and Gauss-Seidel iterative methods, Ding and his coworkers recently presented a large family of the least squares based iterative methods for solving the matrix equations and [47–49]. It has been proved that these least squares based iterative solutions always converge fast to the exact ones as long as the unique solutions exist. But the range of the convergence factor is still open. Motivated by the importance of this algorithm, we develop a new way to prove the convergence of the least squares based iterative algorithm for the coupled Sylvester matrix equations and . According to the new proof, we obtain the optimal convergence factor of this iterative algorithm and extend the iterative algorithm to solve the generalized Sylvester matrix equation . The algorithm in this paper can be extended to other more general and complex matrix equations [50–52].

The paper is organized as follows. Section 2 gives some preliminaries. Section 3 presents a new proof to the least squares iterative algorithm for solving the coupled Sylvester matrix equations and . Section 4 revises this algorithm to solve the equation . Section 5 gives an example to illustrate the effectiveness of the proposed results. Finally, we offer some concluding remarks in Section 6.

2. Basic Preliminaries

Some symbols and lemmas are introduced first. is the identity matrix of size . is an identity matrix with an appropriate size. denotes the eigenvalue set of matrix . denotes the matrix determinant. denotes the Frobenius norm of and is defined as . For an matrix

denotes an -dimensional vector formed by

represents the Kronecker product of matrices and . A formula involved the vec-operator and the Kronecker product is [53].

If matrix is invertible, then is symmetric and idempotent. To be more specific, we have the following lemma.

Lemma 1. For the symmetric matrix , there exists an orthogonal matrix such that

Moreover, .

3. The Coupled Sylvester Matrix Equations

In this section, we will prove the convergence of the least squares iterative algorithm for solving the following coupled Sylvester matrix equations:

where , , and are known and are to be determined. By using the hierarchical identification principle, the least squares based iterative algorithm for solving (4) is given by [47]

Here, and

To initialize the algorithm, we take and as some small real matrices, for example, with being an matrix whose elements are all 1.

Theorem 2. If (4) has unique solutions and , then for any initial values and the iterative solutions and given by iteration in (5) converge to and ; that is,

or the error matrices and converge to zero; in this case, the optimal convergence factor is .

Proof. Define two error matrices

Using (4) and (5) gives

Expanding (9) gives

Simplifying (10) gives

Using the formula gives

Set

Using these symbols, a compact form of (12) is

Set

The eigenvalues of inside the unit circle will complete the proof. Let be an eigenvalue of ; we will show that , or, if , then .

Consider the following characteristic polynomial of :

It is not hard to show that by calculation. Since

it follows from the formula

that

On the other hand, since (4) has a unique solution, it follows that the matrix

is invertible. A determinant expansion shows that

So it gives . Since

expanding it gives

Setting , it can be rewritten as

According to Lemma 1, there exists an orthogonal matrix such that

Using (25) can be manipulated to get

If , then there exist the nonzero vectors that satisfy , which can be written as

Since and are orthogonal matrices, it follows that

where and . Then from (28), it gives

Using (26) gives

According to (29) and (30), (27) can be manipulated to get

So it gives

Suppose that , where , . From Schur decomposition theorem, there exists a decomposition , where is an orthogonal matrix, , and is a strictly upper triangular matrix. Then from (23), it gives

It follows that . Since , we obtain , . Thus, .

Next, we determine the optimal convergence factor. From and

one gets . Set . Taking absolute values of these eigenvalues, the optimal convergence factor satisfies

Equation (35) is equivalent to . Solving it gives . The proof is completed.

4. The Generalized Sylvester Matrix Equation AXB + CXD = F

In this section, we use iteration in (5) to solve the generalized Sylvester matrix equation. Consider the following equation:

where , , and are given constant matrices and is the unknown matrix to be solved. The following conclusion is obvious.

Equation (36) has a unique solution if and only if

In this case, the unique solution is given by .

Setting and , (36) can be equivalently expressed as

If

then (38) has a unique solution. It is easy to show that if or , then (38) is equivalent to (36). Next, we show that if or , then (39) is equivalent to (37). That is, we have the following determinant result:

According to Theorem 2, (38) can be solved by iteration in (5), and from or , (36) can be solved.

5. Example

In this section, an example is offered to illustrate the convergence of the proposed iterative algorithm.

Example 1. Consider the coupled Sylvester matrix equations in the form of (4) with

The unique solution is found to be

Taking as the initial iterative values and using iteration (5) to compute and , the iterative values of and are shown in Table 1 with the relative error

The effect of changing the convergence factor is illustrated in Figure 1.

Table 1: The least squares iterative solutions ( = 1.0).

(%)

1
2.41386
5.12921
15.19476
−3.29213
8.34835
1.59243
4.18906
7.37550
58.29997

2
3.78243
4.89986
8.50024
3.36107
7.11113
1.69190
4.40964
7.01546
8.34339

3
3.76982
4.86436
9.81323
3.24974
8.02454
1.93498
4.94575
6.99503
6.82277

4
3.98646
4.99976
8.94082
3.91276
7.87581
1.98403
4.92193
7.02174
1.09451

5
3.96846
4.97726
9.13074
3.92885
8.00448
1.99063
4.99536
6.99758
0.92202

6
3.99906
5.00066
8.99011
3.98870
7.98094
2.00140
4.99013
7.00441
0.15912

7
3.99573
4.99642
9.02215
3.99419
8.00093
1.99865
4.99957
6.99955
0.14124

8
3.99996
5.00016
8.99824
3.99851
7.99694
2.00071
4.99874
7.00081
0.02497

9
3.99940
4.99943
9.00381
3.99973
8.00018
1.99980
4.99996
6.99992
0.02344

10
4.00000
5.00003
8.99969
3.99980
7.99949
2.00018
4.99983
7.00015
0.00413

11
3.99991
4.99991
9.00066
4.00004
8.00003
1.99997
5.00000
6.99999
0.00405

12
4.00000
5.00001
8.99994
3.99997
7.99991
2.00004
4.99998
7.00003
0.00070

Solution
4.00000
5.00000
9.00000
4.00000
8.00000
2.00000
5.00000
7.00000

Figure 1: The relative error versus iteration .

From Table 1 and Figure 1, we find that the relative error goes to zero with the increasing of the iterative times. This shows that the proposed iterative algorithm is effective. In addition, Figure 1 shows that the optimal convergence factor . This indicates that the result of the optimal convergence suggested in this paper is correct.

6. Conclusions

This paper proved the convergence of the least squares based iterative algorithm of the coupled Sylvester matrix equations and , and the proof determined the range of the convergence factor and the optimal convergence factor. The suggested algorithm can also be used to solve the generalized Sylvester equation . An example indicated that the iterative solution given by the least squares based iterative algorithm converges fast to its exact solution under proper conditions.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgment

This work was supported by the National Natural Science Foundation of China (no. 6110218).

References

X. P. Sheng, Y. F. Su, and G. L. Chen, “A modification of minimal residual iterative method to solve linear systems,” Mathematical Problems in Engineering, vol. 2009, Article ID 794589, 9 pages, 2009.

A. J. Liu and G. L. Chen, “On the hermitian positive definite solutions of nonlinear matrix equation ,” Mathematical Problems in Engineeering, vol. 2011, Article ID 163585, 18 pages, 2011.

H. M. Zhang, “Iterative solutions of a set of matrix equations by using the hierarchical identification principle,” Abstract and Applied Analysis, vol. 2014, Article ID 649524, 10 pages, 2014.

H. M. Zhang and F. Ding, “A property of the eigenvalues of the symmetric positive definite matrix and the iterative algorithm for coupled Sylvester matrix equations,” Journal of the Franklin Institute, vol. 351, no. 1, pp. 340–357, 2014.

M. Hochbruck and G. Starke, “Preconditioned Krylov subspace methods for Lyapunov matrix equations,” SIAM Journal on Matrix Analysis and Applications, vol. 16, no. 1, pp. 156–171, 1995.

J. Yu, Q.-W. Wang, and C.-Z. Dong, “(Anti-)Hermitian generalized (Anti-)hamiltonian solution to a system of matrix equations,” Mathematical Problems in Engineering, vol. 2014, Article ID 539215, 13 pages, 2014.

B. Zhou and G.-R. Duan, “On the generalized Sylvester mapping and matrix equations,” Systems & Control Letters, vol. 57, no. 3, pp. 200–208, 2008.

B. Zhou and G.-R. Duan, “An explicit solution to the matrix equation ,” Linear Algebra and Its Applications, vol. 402, no. 1, pp. 345–366, 2005.

B. Zhou and G.-R. Duan, “A new solution to the generalized Sylvester matrix equation ,” Systems & Control Letters, vol. 55, no. 3, pp. 193–198, 2006.

B. Zhou and G.-R. Duan, “Parametric solutions to the generalized Sylvester matrix equation and the regulator equation ,” Asian Journal of Control, vol. 9, no. 4, pp. 475–483, 2007.

B. Zhou and G.-R. Duan, “Solutions to generalized Sylvester matrix equation by Schur decomposition,” International Journal of Systems Science, vol. 38, no. 5, pp. 369–375, 2007.

Q. B. Liu and G. L. Chen, “Convergence analysis of preconditioned AOR iterative method for linear systems,” Mathematical Problems in Engineering, vol. 2010, Article ID 341982, 14 pages, 2010.

Y.-Q. Bai, T.-Z. Huang, and M.-M. Yu, “Convergence of a generalized USOR iterative method for augmented systems,” Mathematical Problems in Engineering, vol. 2013, Article ID 326169, 6 pages, 2013.

M. Dehghan and M. Hajarian, “Two algorithms for finding the Hermitian reflexive and skew-Hermitian solutions of Sylvester matrix equations,” Applied Mathematics Letters, vol. 24, no. 4, pp. 444–449, 2011.

M. Dehghan and M. Hajarian, “Analysis of an iterative algorithm to solve the generalized coupled Sylvester matrix equations,” Applied Mathematical Modelling, vol. 35, no. 7, pp. 3285–3300, 2011.

M. Dehghan and M. Hajarian, “An iterative algorithm for solving a pair of matrix equations , over generalized centro-symmetric matrices,” Computers & Mathematics with Applications, vol. 56, no. 12, pp. 3246–3260, 2008.

M. Dehghan and M. Hajarian, “Finite iterative algorithms for the reflexive and anti-reflexive solutions of the matrix equation ,” Mathematical and Computer Modelling, vol. 49, no. 9-10, pp. 1937–1959, 2009.

M. Dehghan and M. Hajarian, “An efficient algorithm for solving general coupled matrix equations and its application,” Mathematical and Computer Modelling, vol. 51, no. 9-10, pp. 1118–1134, 2010.

F. Ding and T. Chen, “Gradient based iterative algorithms for solving a class of matrix equations,” IEEE Transactions on Automatic Control, vol. 50, no. 8, pp. 1216–1221, 2005.

M. Dehghan and M. Hajarian, “An iterative method for solving the generalized coupled Sylvester matrix equations over generalized bisymmetric matrices,” Applied Mathematical Modelling, vol. 34, no. 3, pp. 639–654, 2010.

F. Ding, X. M. Liu, H. B. Chen, and G. Y. Yao, “Hierarchical gradient based and hierarchical least squares based iterative parameter identification for CARARMA systems,” Signal Processing, vol. 97, pp. 31–39, 2014.

P. Shi, X. L. Luan, and F. Liu, “ filtering for discrete-time systems with stochastic incomplete measurement and mixed delays,” IEEE Transactions on Industrial Electronics, vol. 59, no. 6, pp. 2732–2739, 2012.

F. Ding, “Combined state and least squares parameter estimation algorithms for dynamic systems,” Applied Mathematical Modelling, vol. 38, no. 1, pp. 403–412, 2014.

X. L. Luan, P. Shi, and F. Liu, “Stabilization of networked control systems with random delays,” IEEE Transactions on Industrial Electronics, vol. 58, no. 9, pp. 4323–4330, 2011.

X. L. Luan, S. Y. Zhao, and F. Liu, “ control for discrete-time Markov jump systems with uncertain transition probabilities,” IEEE Transactions on Automatic Control, vol. 58, no. 6, pp. 1566–1572, 2013.

Y. J. Liu, F. Ding, and Y. Shi, “An efficient hierarchical identification method for general dual-rate sampled-data systems,” Automatica, vol. 50, no. 3, pp. 962–970, 2014.

L. Xie and H. Z. Yang, “Interactive parameter estimation for output error moving average systems,” Transactions of the Institute of Measurement and Control, vol. 35, no. 1, pp. 34–43, 2013.

F. Ding, “State filtering and parameter identification for state space systems with scarce measurements,” Signal Processing, vol. 104, pp. 369–380, 2014.

J. Ding, F. Ding, X. P. Liu, and G. Liu, “Hierarchical least squares identification for linear SISO systems with dual-rate sampled-data,” IEEE Transactions on Automatic Control, vol. 56, no. 11, pp. 2677–2683, 2011.

L. Xie and H. Z. Yang, “Gradient-based iterative identification for nonuniform sampling output error systems,” Journal of Vibration and Control, vol. 17, no. 3, pp. 471–478, 2011.

D. Q. Wang, “Least squares-based recursive and iterative estimation for output error moving average systems using data filtering,” IET Control Theory & Applications, vol. 5, no. 14, pp. 1648–1657, 2011.

X. G. Liu and J. Lu, “Least squares based iterative identification for a class of multirate systems,” Automatica, vol. 46, no. 3, pp. 549–554, 2010.

Y. J. Liu, J. Sheng, and R. F. Ding, “Convergence of stochastic gradient estimation algorithm for multivariable ARX-like systems,” Computers & Mathematics with Applications, vol. 59, no. 8, pp. 2615–2627, 2010.

J. Ding, C. X. Fan, and J. X. Lin, “Auxiliary model based parameter estimation for dual-rate output error systems with colored noise,” Applied Mathematical Modelling, vol. 37, no. 6, pp. 4051–4058, 2013.

D. Q. Wang, F. Ding, and L. Ximei, “Least squares algorithm for an input nonlinear system with a dynamic subspace state space model,” Nonlinear Dynamics, vol. 75, no. 1-2, pp. 49–61, 2014.

J. H. Li, “Parameter estimation for Hammerstein CARARMA systems based on the Newton iteration,” Applied Mathematics Letters, vol. 26, no. 1, pp. 91–96, 2013.

D. Q. Zhu and M. Kong, “Adaptive fault-tolerant control of non-linear systems: an improved CMAC-based fault learning approach,” International Journal of Control, vol. 80, no. 10, pp. 1576–1594, 2007.

D. Q. Zhu, J. Bai, and Q. Liu, “Multi-fault diagnosis method for sensor system based on PCA,” Sensors, vol. 10, no. 1, pp. 241–253, 2010.

D. Q. Zhu, J. Liu, and S. X. Yang, “Particle swarm optimization approach to thruster fault-tolerant control of unmanned underwater vehicles,” International Journal of Robotics and Automation, vol. 26, no. 3, pp. 426–432, 2011.

J. Ding, L. L. Han, and X. M. Chen, “Time series AR modeling with missing observations based on the polynomial transformation,” Mathematical and Computer Modelling, vol. 51, no. 5-6, pp. 527–536, 2010.

Y. J. Liu, Y. S. Xiao, and X. L. Zhao, “Multi-innovation stochastic gradient algorithm for multiple-input single-output systems using the auxiliary model,” Applied Mathematics and Computation, vol. 215, no. 4, pp. 1477–1483, 2009.

Y. B. Hu, B. L. Liu, Q. Zhou, and C. Yang, “Recursive extended least squares parameter estimation for Wiener nonlinear systems with moving average noises,” Circuits, Systems, and Signal Processing, vol. 33, no. 2, pp. 655–664, 2014.

Y. B. Hu, “Iterative and recursive least squares estimation algorithms for moving average systems,” Simulation Modelling Practice and Theory, vol. 34, pp. 12–19, 2013.

D. Q. Zhu, Q. Liu, and Z. Hu, “Fault-tolerant control algorithm of the manned submarine with multi-thruster based on quantum-behaved particle swarm optimisation,” International Journal of Control, vol. 84, no. 11, pp. 1817–1829, 2011.

D. Q. Zhu, Y. Zhao, and M. Z. Yan, “A bio-inspired neurodynamics based backstepping path-following control of an AUV with ocean current,” International Journal of Robotics and Automation, vol. 27, no. 3, pp. 280–287, 2012.

D. Q. Zhu, H. Huang, and S. X. Yang, “Dynamic task assignment and path planning of multi-auv system based on an improved self-organizing map and velocity synthesis method in 3D underwater workspace,” IEEE Transactions on Cybernetics, vol. 43, no. 2, pp. 504–514, 2013.

F. Ding and T. Chen, “Iterative least-squares solutions of coupled Sylvester matrix equations,” Systems & Control Letters, vol. 54, no. 2, pp. 95–107, 2005.

F. Ding and T. Chen, “On iterative solutions of general coupled matrix equations,” SIAM Journal on Control and Optimization, vol. 44, no. 6, pp. 2269–2284, 2006.

F. Ding, P. X. Liu, and J. Ding, “Iterative solutions of the generalized Sylvester matrix equations by using the hierarchical identification principle,” Applied Mathematics and Computation, vol. 197, no. 1, pp. 41–50, 2008.

L. Xie, J. Ding, and F. Ding, “Gradient based iterative solutions for general linear matrix equations,” Computers & Mathematics with Applications, vol. 58, no. 7, pp. 1441–1448, 2009.

J. Ding, Y. J. Liu, and F. Ding, “Iterative solutions to matrix equations of the form ,” Computers & Mathematics with Applications, vol. 59, no. 11, pp. 3500–3507, 2010.

L. Xie, Y. J. Liu, and H. Z. Yang, “Gradient based and least squares based iterative algorithms for matrix equations ,” Applied Mathematics and Computation, vol. 217, no. 5, pp. 2191–2199, 2010.

H. M. Zhang and F. Ding, “On the Kronecker products and their applications,” Journal of Applied Mathematics, vol. 2013, Article ID 296185, 8 pages, 2013.