Convergence Results on Iteration Algorithms to Linear Systems

In order to solve the large scale linear systems, backward and Jacobi iteration algorithms are employed. The convergence is the most important issue. In this paper, a unified backward iterative matrix is proposed. It shows that some well-known iterative algorithms can be deduced with it. The most important result is that the convergence results have been proved. Firstly, the spectral radius of the Jacobi iterative matrix is positive and the one of backward iterative matrix is strongly positive (lager than a positive constant). Secondly, the mentioned two iterations have the same convergence results (convergence or divergence simultaneously). Finally, some numerical experiments show that the proposed algorithms are correct and have the merit of backward methods.


Introduction
The primal goal of this paper is to study the iterative methods of the linear systems: where is a given × complex or real matrix. It is well known that linear systems arise in studies in many areas such as engineering and industrial science. For example, in the field of numerical solutions of differentialalgebraic equations (DAE) and ordinary differential equations (ODE) [1][2][3] it is very important to solve (1). In digital image and signal processing, especially in compressed sensing, Stojnic [4] has mentioned that the systems (1) are the mathematical background of compressed sensing problems and studied sharp lower bounds on the values of allowable sparsity for any given number (proportional to the length of the unknown vector) of equations for the case of the socalled block-sparse unknown vectors. In the blind source separation of signal, Congedo et al. [5] have showed that it is very important to solve (1) and proposed a special method with joint singular value decomposition. In the field of biomedical engineering, Deo et al. [6] have mentioned that the cardiac electrical activity can be described by the bidomain equations and pointed out that the numerical solution of partial differential equations (PDEs) associated with bidomain problems often leads to (1). Moreover, they have proposed a novel preconditioner for the PCG method to solve (1) and a cheap iterative method such as successive overrelaxation (SOR) to further refine the solution for a desired accuracy. In 2008, Shou et al. [7] have showed that the reconstruction of epicardial potentials (EPs) from body surface potentials (BSPs) can be characterized as an ill-posed inverse problem and geometric errors in the ECG inverse problem will directly affect the calculation of transfer matrix in (1). In the field of systems and control science, Ding and Chen [8] have pointed out that Sylvester equations in systems and control especially Lyapunov equations in continuousand discrete-time stability analysis can be converted into equivalent equations as (1). In the field of machine learning many problems of classification and regression, such as single-hidden layer neural networks [9,10], support vector machines, functional neural networks, and so on, can be summarized as (1). Therefore, the solution of (1) is very important in scientific computing.
The methods to solve linear systems can be roughly divided into two categories: direct methods and iterative methods 1. Iterative methods are more suitable than direct methods for large linear systems [11,12]. The current research on iterative algorithms has been more mature, but how to make it fit the new architecture model is complicated. In order to gain more good performance, acceleration has been applied and architecture has been considered [13].
In this paper, we do some research with the iterative algorithm. To this end, the paper is organized as follows. In Section 2, we introduce the backward MPSD (backward modified preconditioned simultaneous displacement) iterative method which is a unified form of some important backward iterations. In Section 3, we first introduce some important lemmas which will be used and then we obtained the convergence results between backward MPSD iteration and Jacobi iteration. We also proposed convergence results between some backward iterations and Jacobi iteration in the corollaries. In Section 4, some examples and numerical experiments have been done to make sure of the correctness of results. Especially, we point out that the backward iteration is better than the original one in many cases just like Example 4.

A Unified Framework of Iteration Matrix and Algorithm
The basic idea to solve (1) is matrix splitting. If we let where = diag( ) is a diagonal matrix obtained from and nonsingular and and are strictly lower and upper triangular matrices obtained from , (1) becomes the equivalent one: At this moment, −1 = ( − − ). The Jacobi iterative matrix is The MPSD (modified preconditioned simultaneous displacement) iterative method is studied in [14][15][16][17].
If > 0 is a real constant, obviously (3) is equivalent to At the same time, if 1 , 2 are real constants, we can obtain the following equivalent from (5): It is easy to verify that With (7), we can construct the backward MPSD iterative method as follows: wherẽ, which we named as backward MPSD iterative matrix. Also, we have the following algorithm.

Remark 1.
With special values of 1 , 2 , and , we have the following.
The convergence relationship between the Gauss-Seidel iterative matrix and the Jacobi iterative matrix is studied in [12], and the generalized results are studied in [18]. Some eigenvalue relationships between other iterative matrices and Jacobi iterative matrix are studied with the p-cyclic case in The Scientific World Journal 3 [19][20][21][22][23][24][25][26]. Some backward iterations are studied in [27]. In the following we consider the convergence results between the backward MPSD iterative matrix and the Jacobi iterative matrix and obtain convergence relationships between some other backward iterative matrices and Jacobi matrix.

Convergence Results
In order to obtain the convergence results, we give some wellknown results which will be used in the proof of Theorem 7 as follows.
Definition 2 (see [13]). The splitting = − with and nonsingular is called a regular splitting if −1 ≥ 0 and ≥ 0. It is called a weak regular splitting if −1 ≥ 0 and It is obvious that a regular splitting is a weak regular splitting.

(3) ( ) increases when any entry of increases,
Lemma 6 (see [12]). Let = ( ) ≥ 0 be an irreducible × matrix. Then for any > 0, either By the lemmas above, we give the convergence theorem in the following.
(2) One and only one of the following mutually exclusive relations is valid.
Thus, The Jacobi iterative method and the backward MPSD iterative method are either both convergent or both divergent.
(2) One and only one of the following mutually exclusive relations is valid.
The Scientific World Journal Thus, The Jacobi iterative method and the backward JOR iterative method are either both convergent or both divergent.
(2) One and only one of the following mutually exclusive relations is valid.
Thus, The Jacobi iterative method and the backward Gauss-Seidel iterative method are either both convergent or both divergent.  Thus, The Jacobi iterative method and the backward SOR iterative method are either both convergent or both divergent. Corollary 11. Let the coefficient matrix of (1) be irreducible, = + ≥ 0 the Jacobi matrix, and̃, ,0 the backward AOR iterative matrix. Then, for 0 ≤ < ≤ 1, we have the following.
(2) One and only one of the following mutually exclusive relations is valid.
Thus, The Jacobi iterative method and the backward AOR iterative method are either both convergent or both divergent.
(2) One and only one of the following mutually exclusive relations is valid.
Thus, The Jacobi iterative method and the backward SSOR iterative method are either both convergent or both divergent.
(2) One and only one of the following mutually exclusive relations is valid.
Thus, The Jacobi iterative method and the backward EMA iterative method are either both convergent or both divergent.
(2) One and only one of the following mutually exclusive relations is valid.
Thus, The Jacobi iterative method and the backward PSD iterative method are either both convergent or both divergent.
(2) One and only one of the following mutually exclusive relations is valid.
Thus, The Jacobi iterative method and the backward PJ iterative method are either both convergent or both divergent.
Remark 16. The convergence results between the backward MPSD and Jacobi iterative matrix are proposed, and The convergence results between some special cases of backward MPSD (such as backward JOR, backward G-S, backward EMA, and backward PSD) and Jacobi iterative matrix are obtained. These results involve some special iterative methods which are proposed in the references.

Numerical Examples
In this section, we show five examples. The first three examples are used to show the convergence of the proposed iterative methods. Example 4 is used to show the divergence of the proposed iterative methods. Example 5 shows that the backward iterative methods are better than the origin methods when the upper triangular part dominates the lower triangular part. In the following figures, horizontal axis denotes the numbers of iterations and vertical axis denotes the errors of iterations. Example 1. Let the coefficient matrix and the vector of (1) be ] .
The Jacobi iterative matrix is ] .
under a uniform square mesh of five-point difference approximations, and the interior mesh points as shown in Figure 1 [28], we can obtain the linear system (1), where the matrix and the vector of (1) are  ] .
(1) Let = = 1/2, 1 = 2 = = 1/4. We obtain the backward PSD iterative matrix̃, , =̃1 /2,1/4,1/4 , and From Figures 2, 3, 4, and 5, the errors of Jacobi iteration are denoted by blue circles and that of MPSD iteration is denoted by red stars. By the figures above, We know that  Jacobi iteration is better than backward PSD, JOR, and EMA iteration and is worse than PJ iteration under the values of 1 , 2 , and in this example.
It shows that the backward MPSD iteration is invalid for this example.
From Figure 6, the red stars denote the error of backward Guass-Seidel iteration and the blue circles denote that of Guass-Seidel. So, the backward iterative methods are better than the original methods under the assumption that the upper triangular part dominates the lower triangular part.

Conclusions
The Jacobi iteration is the basic iteration for linear systems and easier to the analysis of the convergence than other iterations. In the paper, we proposed the backward MPSD iteration and obtained the convergence result between backward MPSD iteration (including iterations such as backward JOR, backward G-S, backward EMA, and backward PSD) and Jacobi iteration. We pointed out that the backward MPSD iteration and the Jacobi iteration are either both convergent or both divergent under the assumptions in Theorem 7. So, we can ascertain the convergence or divergence of backward MPSD iteration by Jacobi iteration. In some case, the backward iteration is better than the original one.