A Modification of Minimal Residual Iterative Method to Solve Linear Systems

We give a modification of minimal residual iteration MR , which is 1V-DSMR to solve the linear system Ax b. By analyzing, we find the modifiable iteration to be a projection technique; moreover, the modification of which gives a better at least the same reduction of the residual error than MR. In the end, a numerical example is given to demonstrate the reduction of the residual error between the 1V-DSMR and MR.


Introduction
One of the important computational problems in the applied science and engineering is the solution of n × n nonsingular linear systems of equations: Ax b, 1.1 where A ∈ R n×n is a symmetric positive definite matrix referred to as an SPD matrix , b ∈ R n is given, and x ∈ R n is unknown.To solve this problem, usually an iterative method is spurred by demands, which can be found in excellent papers 1, 2 .Most of the existing practical iterative techniques for solving larger linear systems of 1.1 utilize a projection process in one way or another; see, for example, 3-9 .
Projection techniques are presented in different forms in many other areas of scientific computing, which can be formulated in abstract Hilbert functional spaces and finite element spaces.Furthermore, projection techniques are the process in which one attempts to solve a set of equations by solving each separate equation by correcting so that it is small in some norm.The idea of projection process is to extract an approximate solution to 1.1 from a subspace of R n .Denote K and L the search subspace and the constraints subspace, respectively.Let m be their dimension and x 0 ∈ R n be an initial guess to the solution of 1.1 .A projection method onto the subspace K and orthogonal to L is a process which finds an approximation solution x ∈ R n to 1.1 by imposing the Petrov-Galerkin conditions that x belongs to the affine space x 0 K and the new residual vector is orthogonal to L, that is, From this point of view, the basic iterative methods for solving 1.1 , such as Gauss-Seidel Iteration GS , Steepest Descent Iteration SD , Minimal Residual Iteration MR , and Residual Norm Steepest Descent Iteration RNSD , all can be viewed as a special case of the projection techniques.
In 2 , Ujević obtained a new iterative method for solving 1.1 , which is considered as a modification of Gauss-Seidel method.In 10 , Jing and Huang pointed that this iterative method is also a projection process and named this method as "one-dimensional double successive projection method" referred to as 1D-DSPM .In the same paper 10 , the authors obtained another iterative method, which is named as "two-dimensional double successive projection method" referred to as 2D-DSPM .The theory indicates that 2D-DSPM gives a better reduction of error than 1D-DSPM.

Notations and Preliminaries
In this paper, we will consider the following linear system of equations: where A ∈ R n×n is not a symmetric but a positive definite matrix of order n, b ∈ R n is a given element, and x unknown.For linear systems 2.1 , we can use the classical minimal residual iteration to solve, which can be found in 11 .Here, we will give a modification of minimal residual iteration to solve linear system 2.1 .We call the modification method as one vector double successive MR abbreviated 1V-DSMR and compare reduction of the residual error at step k 1 between the modification iteration and original MR.Hence, we find that the modification iteration gives a better reduction of the residual error than the original MR.
x, y y T x denotes a vector inner product between the vector x, y ∈ R n .We define the inner products as

2.3
In this subsection, we will recall minimal residual iterative method and give some properties of this iteration.
For the linear system 2.1 , we can use the following algorithm which is called minimal residual iteration, viewed in 11 .

Algorithm 2.1. 1 Choose an initial guess solution x
3 If r k 0, then stop; else, 4 calculate
The minimal residual iteration can be interpreted with projection techniques.Here we represent the principles of this method in our uniform notation as follows: where α p/a.If we choose K span{v 1 } and L span{Av 1 }, then 1.2 turns to find where x k 1 x k αv 1 .Equation 2.7 can be represented in terms of inner product as giving rise to α p/a, which is the same as in 2.6 .
If we choose a special v 1 r k , then 2.6 is the minimal residual iteration MR ; up to now, it is clear that MR is a special case of projection methods.
For the MR, we have the following property.Proof.Using 2.6 , we obtain

2.11
where v 1 r k .Hence we have

2.12
From Lemma 2.2, easily, we can get the reduction of the residual error of MR as follows: 2.13

An Interpretation of 1V-DSMR with Projection Technique
In this section, we will give the modification of the minimal residual iterative method, which is abbreviated to 1V-DSMR; we can present this method in our uniform notation as follows: where α p/a and β aq − cp /ad.We will have a two-step investigation of 1V-DSMR.The first step is to choose K 1 span{v 1 } and L 1 AK 1 span{Av 1 }, then it turns into the proceeding of MR, so we have α p/a.The next step is a similar way to choose K 2 span{v 2 } and L 2 AK 2 span{Av 2 }; denote x k 1 x k αv 1 and 1.2 turns to find where x k 1 x k 1 βv 2 .Equation 3.2 can be represented as in terms of inner product as

3.4
This gives rise to β aq − cp /ad, which is the same as in 3.1 .If we choose a special v 1 r k , then 3.1 is a modification of the minimal residual iteration, which is named as 1V-DSMR; up to now, it is clear that 1V-DSMR is also a special case of projection methods.
As 1V-DSMR, we have the following relation of residual errors.
where r k is the same as in Lemma 2.2, and a, c, d, p, q are defined in 2.2 and 2.3 .
Proof.Using 3.1 , we have

3.6
By deduction, we get

6 Mathematical Problems in Engineering
If we substitute α p/a and β aq − cp /ad into 3.7 , then we obtain From Theorem 3.1, we also get a reduction of residual error of 1V-DSMR as follows: 3.9 Next we will depict the comparison results with respect to residual error reduction between 1V-DSMR and MR.

Theorem 3.2. 1V-DSMR gives a better (at least the same) reduction of the residual error than MR.
Proof.From the equalities 2.13 and 3.9 , we have which proves the assertion of Theorem 3.2.
Theorem 3.2 implies that the residual error of MR is bigger than that of 1V-DSMR at k 1 step if the residual vectors r k and r k at the kth iteration are equal to each other.

A Particular Method of 1V-DSMR
In this section, particular v 1 and v 2 will be chosen, and an algorithm to interpret 1V-DSMR is obtained.
Since 1V-DSMR is a modification of minimal residual iteration, take v 1 r k .In general, v 2 may be chosen in different ways.Here, we choose a particular v 2 x k−1 , then from 3.1 , each step of 1V-DSMR is as follows: where r −1 0 and

4.2
In this case, the first step of 1V-DSMR is the same as minimal residual iteration.

Mathematical Problems in Engineering 7
The above results provide the following algorithm of 1V-DSMR.
Algorithm 4.1 A particular implementation of 1V-DSMR in a generalized way . 1 Choose an initial guess solution x 0 of 2.1 , k : 0. 2 Calculate 4.4 4 If r k 0, then stop; else, 5 calculate Proof.From the equality 3.9 , we get

4.6
This means that the sequence r k 2 is a decreasing and bounded one.Thus, the sequence in question is convergent implying that the left-hand side tends to zero.Obviously, r k 2 tends to zero, and the proof is complete.
Mathematical Problems in Engineering

Numerical Examples
In this section, we use examples to further examine the effectiveness and show the advantages of 1V-DSMR over MR.We compare the numerical behavior of Algorithm 4.1 with Algorithm 2.1.All the tests are performed by MATLAB 7.0.Because of the influence of the error of roundoff, we regard the matrix A as zero matrix if A < 10 −10 .
For convenience of comparison, consider the two-dimensional partial differential equations on the unit square region Ω 0, 1 × 0, 1 of the form − pu x x − qu y y ru x ru x su y su y tu f, 5.1 where p, q, r, s, and t are all given real valued function of x and y, which are as follows: p e −xy , q e xy , r 20 x y , s 10 x y , and t 1/ 1 x y .
Here, we use a five-point finite difference scheme to discretize the above problem with a uninform grid of mesh spacing Δx Δy 1/m in x and y directions, respectively; we can obtain a matrix of order m × m as m varies, which is called PDE matrix.Now, we take m 30, then we get a matrix, which is called PDE900, and denoted by P .It is easy to check that P is real unsymmetrical and nonsingular.
If we take the coefficient matrix A of linear system 2.1 as P , the right vector b 1, 1, . . ., 1 T , and initial iterative vector x 0 b, then we use Algorithms 2.1 and 4.1 to compute the linear system 2.1 , respectively.The comparison results between MR and 1V-DSMR are shown in Figure 1.
From Figure 1, we can see that the convergence velocity of Algorithm 4.1 is always faster than that of Algorithm 2.1.In fact, when we use Algorithm 2.1 to compute the linear system 2.1 , we only need to iterate 814 steps, and the residual norm is r 814 ≤ 9.8625×10 −11 .

Theorem 3 . 1 .
Let { x k 1 } generated by 3.1 and r k 1 b − A x k 1 , then we have

4 . 5 6Theorem 4 . 2 .
Goto step 4 .About Algorithm 4.1, we have the following basic property.If A is a positive matrix, then the sequence of the iterations of Algorithm 4.1 converges to the solution of the linear system Ax b.

Figure 1 :
Figure 1: Comparison of convergence curve of residual norm 1.1 between Algorithm 2.1 and Algorithm 4.1.
Let {x k } and {r k } be generated by Algorithm 2.1, then we have