JAM Journal of Applied Mathematics 1687-0042 1110-757X Hindawi 10.1155/2020/9874162 9874162 Research Article Some Hyperbolic Iterative Methods for Linear Systems https://orcid.org/0000-0001-5788-0078 Niazi Asil K. https://orcid.org/0000-0002-6248-697X Ghasemi Kamalvand M. Wang Qing-Wen Department of Mathematics Lorestan University Khorramabad Iran lu.ac.ir 2020 712020 2020 18 10 2019 30 11 2019 03 12 2019 712020 2020 Copyright © 2020 K. Niazi Asil and M. Ghasemi Kamalvand. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

The indefinite inner product defined by J=diagj1,,jn,jk1,+1, arises frequently in some applications, such as the theory of relativity and the research of the polarized light. This indefinite scalar product is referred to as hyperbolic inner product. In this paper, we introduce three indefinite iterative methods: indefinite Arnoldi’s method, indefinite Lanczos method (ILM), and indefinite full orthogonalization method (IFOM). The indefinite Arnoldi’s method is introduced as a process that constructs a J-orthonormal basis for the nondegenerated Krylov subspace. The ILM method is introduced as a special case of the indefinite Arnoldi’s method for J-Hermitian matrices. IFOM is mentioned as a process for solving linear systems of equations with J-Hermitian coefficient matrices. Finally, by providing numerical examples, the FOM, IFOM, and ILM processes have been compared with each other in terms of the required time for solving linear systems and also from the point of the number of iterations.

1. Introduction

Nowadays, iterative methods are used extensively for solving general large sparse linear systems in many areas of scientific computing because they are easier to implement efficiently on high-performance computers than direct methods.

Projection methods for solving systems of linear equations have been known for some time. The initial development was done by A. de la Garza .

One process by which an approximate solution x˜ of the linear system Ax=b can be found is a projection method onto the subspace K and orthogonal to . This method focuses on this requirement that x˜ belongs to K and the new residual vector bAx˜ be orthogonal to (for more details, refer to ). Around the early 1950s, the idea of the Krylov subspace iteration was established by Cornelius Lanczos and Walter Arnoldi. Lanczos’s method was based on two mutually orthogonal vector sequences and his motivation came from eigenvalue problems. In that context, the most prominent feature of the method is that it reduces the original matrix to tridiagonal form. Lanczos later applied his method to solve linear systems, in particular, symmetric ones. Krylov subspace iterations or Krylov subspace methods are iterative methods which are used as linear system solvers and also iterative solvers of eigenvalue problems. Krylov subspace methods which built up Krylov subspaces look for good approximations to eigenvectors. It is done by keeping all computed approximates and by combining them for a better solution.

In this paper, we introduce three iterative methods in the space with hyperbolic inner product. These methods are indefinite Arnoldi, indefinite Lanczos (ILM), and indefinite full orthogonalization (IFOM), and we define new algorithms to run these hyperbolic versions. By the numerical examples, we will compare these indefinite algorithms with their common definite modes, from the point of the number of iterations and the required time to run the algorithms.

This paper is organized as follows: in Section 2, the indefinite Arnoldi’s process is proposed to construct a J-orthonormal basis. In Section 3, we present IFOM to solve the linear system of equations, and in Section 4, we give the ILM. At the end of this section, several numerical examples are expressed to compare the run speed and the number of repetitions. Counting the arithmetic act of multiplication in FOM, IFOM, and ILM algorithms and conclusion are the last two sections, respectively.

2. Indefinite Arnoldi’s Method

We know that there are many applications which require a nonstandard scalar product which is usually defined by x,yJ=yJx, where J is some nonsingular matrix and many of these applications consider Hermitian or skew-Hermitian J and an indefinite scalar product is defined by nonsingular Hermitian indefinite matrix Jn×n as x,yJ=yJx. When J is the signature matrix J=diag±1=diagj11,j22,,jnn where jkk1,1 for all k, the scalar product is referred to as hyperbolic and takes the following form:(1)x,yJ=yJx=i=1njiixiyi¯.

Example 1.

Let x=x1,,xnT, y1,,ynTn and define(2)x,y=i=1rxσiy¯σii=r+1nxσiy¯σi,where σ is a permutation for which σi=ji and ji1,,n and r is an arbitrary integer from 0 to n. It is easy to see that .,. is an indefinite inner product and its corresponding nonsingular Hermitian matrix is in the form J=diag±1, wherein r is the number of +1 and nr is the number of 1. Because if(3)x,y=Jx,y,then(4)x,y=Jx,y=yJx=y¯1,,y¯nTJx1,,xn=i=1rxσiy¯σii=r+1nxσiy¯σi,and conversely, if(5)x,y=i=1rxσiy¯σii=r+1nxσiy¯σi,then it is clear that x,y=Jx,y, for all x,yn.

Definition 1.

If M is any nonzero subspace of n, then the basis x1,,xk for M is said to be an orthogonal basis with respect to the indefinite inner product .,. if xi,xj=0 for ij, and is said to be an orthonormal basis if in addition to orthogonality, xi,xi=±1 for i=1,,k. For the hyperbolic inner product, the above definitions of orthogonal basis and orthonormal basis are said to be J-orthogonal and J-orthonormal bases, respectively.

Let Cn be a vector space with an indefinite inner product .,.. A vector yCn is called nonneutral if y,y0. Note first of all that any set of nonneutral vectors y1,,ym which is orthogonal in the sense of the indefinite inner product .,. is necessarily linear independent. To see this, suppose that j=1mgjyj=0, and hence, for k=1,,m,(6)j=1mgjyj,yk=gkyk,yk=0.

Then, it follows that gk=0.

In this section, we construct the indefinite Arnoldi’s method and then we turn it into a practical algorithm.

Definition 2.

Let a matrix A and a starting vector v be given. Then, the m-dimensional Krylov subspace KmA,V is spanned by a sequence of m column vectors:(7)KmA,Vspanv,Av,A2v,,Am1v.

It is well known that the construction of a basis with Arnoldi’s method for the Krylov subspace KmA,V leads to an upper Hessenberg matrix that describes the relation between the basis vectors. Each new basis vector can be constructed from the existing set as(8)hj+1,jvj+1=Avji=1jhi,jvi,where hi,j follows from the orthogonality requirement vivj+1=0 and hj+1,j follows from the requirement that vj+12=1. The expression (8) can be formulated in matrix notation as(9)AVm=VmHm+wmemT,where Hm is an upper Hessenberg matrix. For further details, refer to . Now, the purpose of this section is to construct a J-orthonormal basis for the Krylov subspace (7) and the indefinite Arnoldi’s process is an algorithm that brought us to this goal. Furthermore, this process is a tool that condensed the matrix into a Hessenberg form. The indefinite Arnoldi’s algorithm for the computation of a J-orthonormal basis of the Krylov (7) subspace is shown in Algorithm 1.

<bold>Algorithm 1:</bold> Indefinite Arnoldi’s algorithm.

Choose a vector x such that x,x0

Define v1=x/x,x

For j=1,,m Do:

For i=1,,j Do:

Compute hijAvj,vi and tvi=vi,vi

Compute wjAvji=1jtvihijvi

α = w j , w j , if α=0 then stop

v j + 1 = w j / α

t v j + 1 = v j + 1 , v j + 1

h j + 1 , j = t v j + 1 α

EndDo

EndDo

Proposition 1.

Assume that the indefinite Arnoldi’s algorithm does not stop before the m-th step. Then, the vectors v1,,vm form a J-orthonormal basis of the Krylov subspace KmA,v.

Proof.

By considering the following expression, the proof is straightforward:(10)tvj+1hj+1,jvj+1=Avji=1jtvihijvi,j=1,,m.

Proposition 2.

Define

Hm, the m×m Hessenberg matrix whose nonzero entries hij are defined by indefinite Arnoldi’s algorithm

Vm, the n×m matrix with column vectors v1,,vm

J=diagtv1,,tvm

Then, the following relations are valid:(11)AVm=VmJ´Hm+tvm+1hm+1,mvm+1emT,(12)VmJAVm=Hm.

In particular, if m=n, then(13)Vn1AVn=J´Hn.

Proof.

Indeed, in general, (11) is the matrix representation of (10):(14)AVm=VmJ´Hm+wmemT=VmJ´Hm+tvm+1hm+1,mvm+1emT=VmJ´vm+1Hm0,,0,tvm+1hm+1,m.

Now, to see (12), left-multiply relation (14) by VmJ. We earn(15)VmJAVm=VmJVmJ´Hm+tvm+1hm+1,mVmJvm+1emT.

On the other hand, given that the vectors v1,,vm build a J-orthonormal basis, then,

According to the definition of vm+1, vm+1=0 or it is orthogonal to v1,,vm, i.e., vm+1,vi=viJvm+1=0, for i=1,,m. Thus,

(16)VmJvm+1=0.

We have

(17)VmJVm=vi,vji,j=diagv1,v1,,vm,vm=J´.

In other words, J´VmJVm=I.

Therefore, relation (15) can be summarized as follows:(18)VmJAVm=Hm,and by left-multiplying by J´, we have(19)J´VmJAVm=J´Hm.

Particularly, if m=n, relation (17) yields that J´VnJ=Vn1 and thereby,(20)Vn1AVn=J´Hn.

It is noteworthy that these concepts are used in  to solve an eigenvalue problem.

3. Indefinite Full Orthogonalization Method

The purpose of this section is to build an algorithm to solve the linear system:(21)Ax=b,where A is an n×n complex matrix and the inner product of the space is hyperbolic. Let K and be two m-dimensional subspace of n. As mentioned in Section 1, one process by which an approximate solution x˜ of the linear system (21) can be found is a projection method onto the subspace K and orthogonal to . This method focuses on this requirement that x˜ belongs to K and the new residual vector bAx˜ be orthogonal to . Suppose that Vm=v1,,vm is an n×m matrix whose columns constitute a J-orthonormal basis of the nondegenerated subspace K and, similarly, Wm=w1,,wm is an n×m matrix whose columns form a J-orthonormal basis of the nondegenerated subspace and suppose that the approximate solution of (21) is as follows:(22)x˜=x0+Vmy,where x0 is an initial guess to the solution of (21). Then, for each i=1,,m, we should have(23)bAx˜,where indicates orthogonality under the hyperbolic product .,.J. Thus,(24)bAx˜wi,for i=1,,m,bAx˜,wi=0bAx0+Vmy,wi=0,and by letting r0=bAx0, we have(25)r0AVmy,wi=0wiJr0=wiJAVmy.

This leads to WmJr0=WmJAVmy. Now, assuming that the m×m matrix WmJAVm is nonsingular, then(26)y=WmJAVm1WmJr0,and therewith, for approximate solution x˜, we earn(27)x˜=x0+VmWmJAVm1WmJr0.

Now, if Vm=Wm=KmA,r0, the indefinite full orthogonalization method (IFOM) is a process which seeks for an approximation solution x˜ from the subspace x0+Km with the proviso that(28)bAx˜Km.

Thus, relation (26) changes to y=VmJAVm1VmJr0. And, by relation (12), it is equal to y=Hm1VmJr0. On the other hand, by defining v1=r0/β where, β=r0,r0, we have VmJr0=VmJβv1=tv1βe1. Thereupon,(29)y=Hm1tv1βe1,and finally,(30)x˜=x0+VmHm1tv1βe1.

Our explanations are summarized in Algorithm 2.

<bold>Algorithm 2:</bold> IFOM algorithm.

Compute r0=bAx0, βr0,r0, and v1r0/β and tv1=v1,v1

Define the m×m matrix Hm=hiji,j=1,,m; set Hm=0

For j=1,,m Do:

Compute wjAvj

For i=1,,j Do:

h i j = w j , v i

w j w j t v i h i j v i

EndDo

α = w j , w j , compute vj+1=wj/α

t v j + 1 = v j + 1 , v j + 1

Compute hj+1,j=tvj+1α. If α=0, set mj and Goto 12

EndDo

Compute ym=Hm1tv1βe1 and xm=x0+Vmym

Proposition 3.

The residual vector of the approximate solution xm calculated by the IFOM algorithm is such that(31)bAxm=tvm+1hm+1,memTymvm+1.

Proof.

By using relations (11) and (29), we have the following relations:(32)bAxm=bAx0+Vmym=r0AVmym=βv1VmJ´Hmymtvm+1hm+1,memTymvm+1=βv1VmJ´HmHm1tv1βe1tvm+1hm+1,memTymvm+1=βv1βv1tvm+1hm+1,memTymvm+1=tvm+1hm+1,memTymvm+1.

We expect that FOM performs better than IFOM in terms of the number of iterations and the required time to run because in the IFOM method, the product of entries of J´ in the entries of H is added to the calculations, when compared to the FOM method. The following example verifies this expectation. In this example, we have used n instead of m, i.e., we have assumed that n is the maximum number of iterations of the algorithm. But instead, we have considered this requirement that Ax^b<108.

Example 2.

Let A is a 150×150 tridiagonal matrix and an arbitrary diagonal matrix J is J=diagj1,,j150,ji+1,1 and suppose that b and v0 are two vectors in 150 (see Figure 1). Suppose that the entries of these vectors and the nonzero entries of A have been randomly selected from zero to five. Then, the effects of the FOM and IFOM algorithms are as follows:

The IFOM algorithm (Algorithm 2) after 149 iterations and within t=1.89 seconds brings the linear system Ax=b to the following condition:

(33)Ax^b<108.

The FOM algorithm does the same with 149 replications and at t=0.6 seconds.

Despite the superiority of the FOM on the IFOM, there is an important property of the IFOM algorithm that is shown in the next section.

4. Indefinite Lanczos Method

This section is devoted to the indefinite Lanczos method. As can be seen in the following, this method is expressed as a special case of the indefinite Arnoldi’s method in the complex space for the special case when the matrix A is J-Hermitian.

Definition 3.

A matrix A is said to be J-Hermitian (J-symmetric) when A=JAJ (A=JATJ) and we write A=AH (A=AT).

Proposition 4.

Assume that indefinite Arnoldi’s method is applied to a J-Hermitian matrix A. Then, the matrix Hm obtained from the process is tridiagonal and Hermitian.

Proof.

From the indefinite Arnoldi’s method, we have relation (12):(34)VmJAVm=Hm.

Thus, VmAJVm=Hm. On the other hand, that A is J-Hermitian yields JA=AJ. Thus, Hm=Hm.

Therefore, the resulting matrix Hm by the indefinite Arnoldi’s algorithm (Algorithm 1) for J-Hermitian matrix A is an upper Hessenberg and Hermitian matrix. In other words, Hm is a Hermitian tridiagonal matrix. The resulting Hm matrix is shown by Tm, and the diagonal elements are denoted by αjhjj, and the off-diagonal elements are denoted by βjhj1,j=hj,j1,(35)Tm=α1β2β2α2β3βmβmαm.

In fact, we have the following:(36)wj=Avji=1jtvihijvi,and the above relation turns to(37)wj=Avjtvj1βjvj1tvjαjvj.

Thus,(38)wj,wjvj+1=Avjtvj1βjvj1tvjαjvj.

By using the hyperbolic inner product both sides in vj+1, we earn(39)tvj+1wj,wj=Avj,vj+1βj+1=tvj+1wj,wj.

This implies that βj+1=βj+1¯; therefore, Tm is in the above form.

With this explanation, the hyperbolic version of the Hermitian Lanczos algorithm can be formulated as given in Algorithm 3.

Now, consider the linear system Ax=b for which A is a J-Hermitian matrix and x0 is an initial vector and the indefinite Lanczos vectors v1,,vm together with the tridiagonal matrix Tm are given. Then, the approximate solution obtained from an indefinite orthogonal projection method on to Km, similar to what was seen for the indefinite Arnoldi’s method, is given by(40)xm=x0+Vmym,ym=Tm1βe1.

Thus, using the above algorithm, Algorithm 4 can be considered as the indefinite Lanczos algorithm for solving a real linear system with the J-Hermitian coefficient matrix.

Similar to what has already been proven for the IFOM algorithm, here also it can be seen that(41)bAxm=βm+1emTymvm+1.

The advantage of ILM (the indefinite Lanczos method) is that it solves some classes of linear systems with different coefficient matrices, for different choices of matrix J. The following examples explain more in which we use n instead of m. In other words, we assume that the maximum number of the iterations of the algorithm is n. Besides, we use this stop condition that Ax^b<108.

<bold>Algorithm 3:</bold>

Choose an initial vector v1 such that tv1=v1,v1=±1

Set β10, v0=0, tv0=0

For j=1,,m; do:

w j A v j t v j 1 β j v j 1

α j w j , v j

w j w j t v j α j v j

α = w j , w j . If α=0, then stop

v j + 1 w j / α

t v j + 1 = v j + 1 , v j + 1

β j + 1 t v j + 1 α

EndDo

<bold>Algorithm 4:</bold>

Compute r0=bAx0, βr0,r0, and v1r0/β

Set β10, v0=0, tv0=0, tv1=v1,v1

For j=1,,m. Do

w j A v j t v j 1 β j v j 1

α j w j , v j

w j w j t v j α j v j

α = w j , w j . If α=0, then stop

v j + 1 w j / α

t v j + 1 = v j + 1 , v j + 1

β j + 1 t v j + 1 α

EndDo

Set Tm=tridiagβi,αi,βi+1, and Vm=v1,,vm.

Compute ym=Tm1tv1βe1, and xm=x0+Vmym.

Example 3.

Consider the linear system Ax=b in which(42)A=A11A12A21A22Mn,where each block Aij is of order n/2×n/2 and A11 and A22 are symmetric matrices and A12T=A21. By choosing J=In/2In/2, it can be seen that A=AT (see Figure 2). Now, let n=200 and suppose that A11 and A22 are diagonal matrices and A12 is a tridiagonal matrix where the diagonal entries of A11, A22, and A12 are chosen arbitrarily in 0,10 and b is a vector in 200 that its entries are selected at this distance (see Figure 3). The performance of the algorithm is as follows:

The IFOM algorithm brings the linear system Ax=b to the condition Ax^b<108, after 123 iterations and within t=1.7 seconds

The FOM does the same with 118 iterations and within t=0.36 seconds

The ILM algorithm does the same with 130 iterations and within t=0.15 seconds

It shows that ILM is more efficient than the IFOM and FOM. However, the number of its iterations is higher but less time is required.

Example 4.

Consider the assumptions of the previous example, except that AMn and A12 is a tridiagonal matrix with complex entries as a+ib in which a0,10 and b1,5 and A12=A21. Then, A=AH and n=200:

By IFOM: the number of iterations is 140, within t=4.06 seconds

By FOM: the number of iterations is 136, within t=1.11 seconds

By ILM: the number of iterations is 154, within t=0.33 seconds

Example 5.

Consider the linear system Ax=b in which(43)A11A12A13A21A22A23A31A32A33Mn,where each block Aij is of order n/3×n/3 and A11, A22, and A33 are symmetric matrices and A12T=A21 and A23=A32T and A13T=A31. By choosing J=In/3In/3In/3, it can be seen that A is J-symmetric, and therefore, it is allowed to apply ILM. Now, suppose that

Aii, i=1,2,3, are diagonal matrices with diagonal entries between zero and ten and A13T=A31=0 and n=300

A12 and A23 are bidiagonal matrices and the entries of the main diagonals and their below entries are chosen between zero and ten

b is a vector in n with entries between zero and ten

x0 is an initial vector in n with arbitrary entries between zero and ten

Then, to achieve the condition Ax^b<108, we need to consider the following:

By IFOM: the number of iterations is 175, within t=5.6 seconds

By FOM: the number of iterations is 167, within t=1.09 seconds

By ILM: the number of iterations is 190, within t=0.39 seconds

As it is seen, the ILM method is superior to FOM and IFOM methods. It is because of the low length of the recursive relation in its algorithm (tree terms for it) (see Figure 4). In other words, we do not need to do orthogonalizations at each step, on the all earlier vectors of its related Krylov subspace.

5. Counting the Arithmetic Act of Multiplication in FOM, IFOM, and ILM Algorithms

For two n-vectors x,y, we have(44)x,y=yJx=i=1njiiyi¯xi,where xi and yi are the i-th elements of the x,y vectors, respectively, and jii is the i,i-th array of J. Thus, 2n multiplication operations are required to perform this indefinite inner product.

Using the above point, the number of multiplication operations required to perform steps (3)–(12) of the IFOM algorithm is equal to(45)mn2+m2+5mn+m2+2m,and the number of multiplication operations required to perform steps (3)–(12) of the FOM algorithm is equal to(46)mn2+m2+5m2n+m22+32m.

However, the number of required multiplication operations to do steps (3)–(11) of the ILM algorithm is(47)mn2+6mn+2m.

Comparison of (46) and (47) shows that, for m12, the number of multiplications of the FOM algorithm is further than the ILM algorithm, and by increasing the value of m, this difference will also increase.

In the aforementioned algorithms, the inverses of the upper Hessenberg matrix Hn or the tridiagonal matrix Tn must be calculated. In this regard, it should be noted that calculating the inverse of the matrix Tm in the ILM algorithm requires less number of multiplications than calculating the inverse of the Hm matrix in the FOM algorithm.

What was said above shows that the run speed of the m steps of ILM is faster than that of the FOM algorithm and this is evident in the preceding section.

6. Conclusion

The indefinite inner product defined by J=diagj1,,jn, jk1,+1, arises frequently in applications. It is used, for example, in the theory of relativity and in the research of the polarized light. More on the applications of such products can be found in . These applications in other fields of science inspired us to resume Lanczos, FOM, and Arnoldi’s methods in the indefinite inner product space. The indefinite Arnoldi’s method is a process that constructs a J-orthonormal basis for the nondegenerated Krylov subspace, and the bases that we proved have a particular common property, about the structure of the product of their vectors. In this paper, IFOM process has been introduced and also a process is made which is useful for solving linear systems of equations with J-Hermitian coefficient matrices. This process is the same Lanczos method that has been restored in the hyperbolic inner product space. In this paper, the FOM, IFOM, and ILM processes have been compared with each other in terms of the time required for solving linear systems and the best one is introduced. Indeed, we show that the run speed of the m steps of ILM is faster than that of FOM and IFOM.

Data Availability

All data used to support the findings of this study are accessible and these data are cited at the relevant places within the text. The only exception is MATLAB codes for drawing the figures of the paper, which are also available from the corresponding author upon request.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

de la Garza A. An iterative method for solving systems of linear equations 1951 Oak Ridge, TN, USA Union Carbide and Carbon Chemicals Company, K-25 Plant Report K-731 Brezinski C. Projection Methods for Systems of Equations 1997 Amsterdam, Netherlands Elsevier Krasnosell’skii M. Vainikko G. Zabreiko P. Rutitskii Y. Stetsenko V. Approximate Solution of Operator Equations 1972 Groningen, Netherlands Wolters-Nordhoff Liesen J. Strakos Z. Krylov Subspace Methods 2013 Oxford, UK Oxford Science Publications Torenbeek R. Vuik K. The Arnoldi and Lanczos methods for approximating the eigenpairs of a matrix 1995 Delft, Netherlands University of Delft Report 96-44 Arnoldi W. E. The principle of minimized iterations in the solution of the matrix eigenvalue problem Quarterly of Applied Mathematics 1951 9 1 17 29 10.1090/qam/42792 Aliyari M. Ghasemi Kamalvand M. A method of indefinite Krylov subspace for eigenvalue problem Mathematical Problems in Engineering 2018 2018 5 2919873 10.1155/2018/29198732-s2.0-85047876412 Kılıçman A. Zhour Z. A. The representation and approximation for the weighted Minkowski inverse in Minkowski space Mathematical and Computer Modelling 2008 47 3-4 363 371 10.1016/j.mcm.2007.03.0312-s2.0-38149024826 Levy B. C. A note on the hyperbolic singular value decomposition Linear Algebra and Its Applications 1998 277 1–3 135 142 10.1016/s0024-3795(98)10055-12-s2.0-0042095075 Onn R. Steinhardt A. O. Bojanczyk A. The hyperbolic singular value decomposition and applications Proceedings of the 32nd Midwest Symposium on Circuits and Systems August 1989 Champaign, IL, USA IEEE 10.1109/MWSCAS.1989.101919 Higham N. J. J-orthogonal matrices: properties and generations SIAM Review 2003 45 3 504 519 10.1137/s00361445024149302-s2.0-0242498756