A new Monte Carlo approach for evaluating the generalized eigenpair
of real symmetric matrices will be proposed. Algorithm for the three smallest eigenpairs
based on the partitioning inverse Monte Carlo iterative (IMCI) method will be considered.

1. Introduction

It is well known that the problem of calculating the largest or smallest generalized eigenvalue problem is one of the most important problems in science and engineering [1, 2]. This problem arises naturally in many applications. Mathematically, it is a generalization of the symmetric eigenvalue problem, and it can be reduced to an equivalent symmetric eigenvalue problem. Let A,B∈ℜn×n be real symmetric matrices and the matrix B a positive definite matrix. Consider the problem of evaluating the eigenvalues of the pencil (A,B), that is, the values for whichAx=λBx.
A generalized eigenvalue problem (1.1) is said to be symmetric positive definite (S/PD) if A is symmetric and B is positive definite.

2. Inverse Vector Iteration Method

Another procedure for eigenvalue prediction is to use the Rayleigh quotient given by [3]λ=μ(x)=xTAxxTBx.
since B is positive definite, then (2.1) is well defined.

Theorem 2.1.

Suppose that λmin=λ1<λ2≤⋯≤λn-1<λn=λmax are n eigenvalues for pencil (A,B) and v1,…,vn, corresponding eigenvectors. Then for arbitrary initial vector x one has [3]
λ1<μ(x)<λn,
where μ(x) is as introduced in (2.1).

Theorem 2.2.

The inverse vector iteration method for arbitrary choice vector x0 is convergent to the smallest eigenvalue and corresponding eigenvector for pencil (A,B). Also, the rate of convergence depends on 𝒪(λ2/λ1)k, where k is number of iterations [3].

Algorithm 1 evaluates the smallest eigenpair based on the inverse vector iteration [4].

<bold>Algorithm 1</bold>

Input:

Initial vector x0

begin

Set λ1(1)=x0TAx0x0TBx0

Forj=0,1,2,…

begin

Solve linear system Azj+1=Bxj for zj+1

Set λ1(j+1)=zj+1TAzj+1zj+1TBzj+1

Set xj+1=zj+1∥zj+1∥

Output: xj,λ1(j)

end

end

3. Monte Carlo Method for Matrix Computations

Suppose that the matrix A∈ℜn×n and two vectors f,h∈ℜn are given. Consider the following Markov chain Ti with length i:Ti:k0⟶k1⋯⟶ki,
where for j=1,…,i,kj∈{1,2,…,n}. The statistical nature of constructing the chain (3.1) follows asp(k0=α)=pα,p(kj=β∣kj-1=α)=pαβ,
where pα and pαβ show the probability of starting chain at α and transition probability from state α to β, respectively.

In fact∑α=1npα=1,∑β=1npαβ=1,α=1,2,…,n.
Define the random variable Wj using the following recursion forW0=1,Wj=Wj-1akj-1kjpkj-1kj,j=1,2,…,i.
Now, define the following random variable:Θ[h]=hk0pk0∑j=0∞Wjfkj.

Theorem 3.1.

Consider the following system:
Ax=b.Let the nonsingular matrix M∈ℜn,such that MA=I-L, then the system (3.6) can be presented in the following form:
x=Lx+f,
where f=Mb. Then under condition Maxi∑j=1n|lij|<1,one has [5]
E{Θ[h]}=〈h,x〉.

Suppose that x(i) is the ith iterative solution of the following recursion relation with x(0)=f. If we set the random variableΘi[h]=hk0pk0∑j=0iWjfkj,
thenE{Θi[h]}=〈h,x(i+1)〉.

By simulating N random paths with length iTi(s):k0(s)⟶k1(s)⟶⋯⟶ki(s),s=1,2,…,N,
we can findΘi(s)(h)=hk0pk0(s)∑j=0iWj(s)fkj,s=1,…,N.
The Monte Carlo estimation can be evaluated byΘi=1N∑s=1NΘi(s)(h)
which is an approximation of 〈h,x(i+1)〉.

From all possible permissible densities, we apply the following:pα=|hα|∑α=1n|hα|,pαβ=|aαβ|∑β=1n|aαβ|,α=1,2,…,n.
The choice of the initial density vector and the transition probability matrix leads to an almost Optimal Monte Carlo (MAO) algorithm.

Theorem 3.2.

Using the above choice p={pα}α=1n and P={pαβ}α,β=1n the variance of the unbiased estimator for obtaining the inverse matrix is minimized [4].

There is a global algorithm that evaluates the solution of system (3.6) for every matrix A. The complexity of algorithm is O(n2lN), where l and N are the average length of Markov chian and the number of simulated paths, respectively [2].

4. Inverse Monte Carlo Iterative Algorithm (IMCI)

Inverse Monte Carlo iterative algorithm can be applied when A is a nonsingular matrix. In this method, we calculate the following functional in each steps:〈Afj,hj〉〈Bfj,hj〉=〈Bfj-1,hj〉〈Bfj,hj〉.
It is more efficient that we first evaluate the inverse matrix using the Monte Carlo algorithm [1, 2, 4]. The algorithm can be realized as in Algorithm 2.

<bold>Algorithm 2</bold>

input:

A∈Rn×n,f0∈Rn

begin

Starting from initial vector f0

Forj=1,2,…

begin

Using global algorithm, calculate the sequence of Monte Carlo

iterations by solving the following system

Afj=Bfj-1

Set

λ(j)=〈Afj,hj〉〈Bfj,hj〉=〈Bfj-1,hj〉〈Bfj,hj〉

Output:

Smallest eigenvector λ1(j), and corresponding eigenvector fj.

end

end

5. Partitioning IMCI

Let the matrix A be partitioned into four blocks A1,A2,A3, and A4, where A1 and A4 are square matrices of order p and q such that p+q=n:A=(A1A2A3A4).
By assumption that all the indicated matrix inversions are realized, it is easy to verify thatA-1=(BLMN),
whereN=(A4-A3A1-1A2)-1,M=-NA3A1-1,L=-A1-1A2N,K=A1-1-A1-1A2M.
Thus inverting a matrix of order n comes down to inverting four matrices, of which two have order p and two have order q, and several matrix multiplications. Therefore the basic Monte Carlo for solving Afj=Bfj-1 will be called as the dimension of matrix Aandequals to threshold. This action causes the convergence acceleration. Now, we can use the following recursion algorithm to obtain the inverse of matrix A (see Algorithm 3).

<bold>Algorithm 3</bold>

Partitioning inverse(S,n)

begin:

n=rank(S);p=n/2

A=S[1:p,1:p];B=S[1:p,p+1:n]

C=S[p+1:n,1:p];D=S[p+1:n,p+1:n]

m= size (A)

if m≤ threshold

AA = Monte Carlo procedure(A)

else begin:

AA = Partitioninginverse(A,m)

N = Partitioninginverse(D-C*AA*B)

M=-N*C*AA;L=-AA*B*N

K=AA-AA*B*M

SS[1:p,1:p]=K;SS[1:p,p+1:n]=L

SS[p+1:n,1:p]=M;SS[p+1:n,p+1:n]=N

end

end

6. Finding More Than One Generalized Eigenvalues

Assume that an eigenvalue λ1 and its corresponding eigenvector v1 have been computed using the partitioning IMCI algorithm. In the first step of the above algorithm, we deflate the matrix A to the matrix B. Then, we repeat again the first step of the algorithm to obtain the dominant eigenvalue of B which is the second dominant eigenvalue of A. Let p values of eigenvalues of pencil (A,B) be computed. Suppose that Vp is a matrix such that the columns of Vp are p vector of eigenvector of pencil (A,B), that is,Vp=[ν1,…,νp],
where νi is eigenvector corresponding eigevalue λi,i=1,2,…,p.

Now, let(Ap,B)=(A+(BVp)Λ(BVp)T,B),
whereΛ=diag{δi-λi},δi>λp,i=1,…,p.
Hence, if we find the pth smallest eigenpair of pencil (A,B), then we can evalute λp+1, that is, (p+1)th smallest eigenvalue of pencil (A,B).

7. Numerical Results

In this section, the experimental results for obtaining the three smallest eigenpairs outlined in Tables 1, 2, and 3. The numerical tests are performed on Intel(R) (Core(TM)2 CPU, 1.83 GHz) personal machine.

Number of chains N=80.

Dimension

Eigenvalues error

Eigenvectors error

λ1(1)

λ1(2)

λ1(3)

v1(1)

v1(2)

v1(3)

64×64

1.26×10-7

4.5×10-7

1.26×10-6

8.36×10-4

.002813

.008154

128×128

1.18×10-7

4.46×10-7

1.33×10-6

6.33×10-4

.002534

.008074

256×256

6.48×10-8

4.37×10-7

1.51×10-6

3.16×10-4

.002151

.008070

512×512

7.91×10-8

2.87×10-7

1.72×10-6

1.83×10-4

.001284

.007759

1024×1024

7.04×10-8

1.99×10-7

1.79×10-6

1.06×10-4

.000712

.00769

The solution when the number of chains increases.

Number of chains

Calculated eignvalues

N

Exact λ1(1)=.74529395

Exact λ1(2)=.85537131

Exact λ1(3)=.91032740

20

.74737713

.85513704

.91008238

40

.74529979

.85537157

.91034304

80

.74529407

.85537175

.9102873

Total computational time for general and partitioning methods.

Dimension

Time (Sec.)

General method

Partitioning method

20×20

.21

.20

40×40

.40

.32

60×60

.71

.54

80×80

1.25

.60

100×100

1.83

.75

120×120

2.54

1.18

140×140

3.67

1.57

160×160

4.42

2.04

180×180

5.58

2.54

200×200

7.20

3.18

8. Conclusion and Future Study

We have seen that Monte Carlo algorithms can be used for finding more than one eigenpair of generalized eigenvalue problems. We analyze the computational complexity, speedup, and efficiency of the algorithm in the case of dealing with sparse matrices. Finally, a new method for computing eigenpairs as the partitioned method is presented. In Figure 1 the comparison of computational times between general Monte Carlo algorithm and partitioning algorithm is shown. The scatter diagram in Figure 2, shows that there is a linear relationship between matrix dimension (equivalently, the number of matrix elements) and total computational time for partitioning IMCI.

Computational times for general and partition methods.

Regression function y=0.0000658008x+0.1048607485.

DimovI.KaraivanovaA.Iterative Monte Carlo algorithm for linear algebra problemDimovI.SaadY.FathiB.A way to obtain Monte Carlo matrix inversion with minimal errorRubinsteinR. Y.