Neural-Network-Based Approach for Extracting Eigenvectors and Eigenvalues of Real Normal Matrices and Some Extension to Real Matrices

,

The following ODE for solving the generalized eigenvalue problem was proposed by [22] d () d =  () −  ( ())  () , where  and  are two real symmetric matrices, and  can be a general form to some degree.Particularly, if  is the identity matrix, (4) can be used to solve the standard eigenvalue problem as (1) and (2).References [16][17][18] extended those neural network based approaches to the case of real antisymmetric or special real matrices of order , where the proposed neural networks can

Main Results
Let  = √ −1 be the imaginary unit,  the conjugate of , and diag[ 1 , . . .,   ] a block diagonal matrix, where   ,  = 1, . . ., , is a square matrix at the th diagonal block.Unless specially stated,  is a real normal matrix in this paper.Lemma 1.In [13],  is a real normal matrix of order  if and only if there exists an orthogonal matrix  such that Here,   ,  = 1, . . ., , are  real eigenvalues of  corresponding to the  real eigenvectors   , and  + ± i + ,  = 1, . . ., , are  pairs of complex eigenvalues of  corresponding to the  pairs of complex eigenvectors   + ± i  + .
For simplicity, let   =   ,   = 0 for  = 1, . . ., .Based on (5) and (6), it is straightforward to verify Then, the following six definitions and two lemmas are presented, which will be involved much in the sequel.
Remark 4. If we randomly choose (0), the projection of (0) on the eigensubspace corresponding to the largest or smallest eigenvalue of  will be nonzero with high probability.Hence, (2) and (3) can almost work well with randomly generated (0).
Proof.Assume that 4 2 + ,  = ,  + 1, . . ., ,  ≥ 1, are the largest eigenvalues of −( −   ) 2 , that is; Based on Lemma 2 and (7), we know that  should be a linear combination of    and    ,  ∈ J 1 .Let In addition, by (8) we have And by ( 9), we have Because  is an orthogonal matrix and for all  ∈ J 1 ,   =  + holds due to (C1), it is straightforward to verify thus proving the lemma.
Proof.Let J 1 and  take the form as (11) and (12), respectively.Based on (13), we can write Following the decomposition of  as (5) and the definition of , we have where Taking ( 12) into (17), we get Based on ( 13) and (19), it is straightforward to verify In addition, since 4 By ( 20) and ( 21), we have By ( 16) and ( 17), we have Then, it is straightforward to verify thus proving the lemma.

Lemma 8. Assume that (C2) holds. Then, 𝑎
Proof.The proof is almost the same to that in Lemma 5.
Note that  1 may be zero; that is, −( −   ) 2 has real eigenvalues.In this case, we have the following lemma.Lemma 9. Assume that (C2) holds and 0 is the smallest eigenvalue of −( −   ) 2 corresponding to the eigenvector  * .Then,  1 is the eigenvalue of  corresponding to the eigenvector  * , where Note that  1 = ⋅ ⋅ ⋅ =   = 0,  ≥ 1, because 0 is the smallest eigenvalue of −( −   ) 2 .Based on the definition of J 2 , we have Applying Lemma 3 to (25), we know that  * should be a linear combination of  1 , . . .,   .Let By ( 8), we have Therefore, Then, by ( 26) and (30), it is straightforward to verify thus proving the lemma.
In the case of  1 ̸ = 0, that is, all the eigenvalues of −( −   ) 2 are complex numbers, we have the following lemma similar to Lemma 7.
Lemma 10.Assume that (C2) holds.Given any nonzero 4 2  1 , the smallest eigenvalue of −( −   ) 2 , and the corresponding eigenvector  * obtained by (25), Proof.The proof is almost the same to that in Lemma 7.

Computing the Eigenvalues with the
Then, based on (9), we get Because  is an orthogonal matrix and for all  ∈ J 3 ,   =  + and thus proving the lemma.
The following lemma introduces an approach for computing a pair of conjugated eigenvectors of  corresponding to the eigenvalues  + ± | + | under the condition (C3).
Without loss of generality, assume that  +1 is the smallest real part of the eigenvalues of  (it may be  1 .However, Lemma 14 has no difference in that case).Applying Lemma 3 to (37), we can obtain 2 +1 , the smallest eigenvalue of (+  ), as well as the corresponding eigenvector, denoted by  * .Then, we have the following lemma.) , ( 3 0 0 0 0 0 4 0 0 0 0 0 5 0 0 0 0 0 5 2 0 0 0 −2 5 ) , only the first three meet (C3), but the last one does not.And only the last three matrices meet (C4), but the first one does not.

Computing the Eigenvalues with the Largest or Smallest
Modulus, as well as the Corresponding Eigenvectors.Reorder the eigenvalues of the symmetric matrix   in ( 9) and the corresponding columns of  such that Without loss of generality, assume that  + ± | + | are the eigenvalues of  that have the largest modulus (it may be   .However, Lemma 16 has no difference in that case), and that  +1 ± | +1 | are the eigenvalues of  that have the smallest modulus (it may be  1 .However, Lemma 17 has no difference in that case).
only the first three meet (C5), but the last one does not.And only the last three meet (C6), but the first one does not.However, there exists some specially constructed  that meet none of (C1) to (C6), for example,  = diag[ 1 ,  2 ,  3 ,  4 ], where Nevertheless, a randomly generated real normal matrix can meet (C1) to (C6) with high probability.

Extension to
where  ∞ = lim  → ∞   is to be a normal matrix with the same eigenvalues as .Such skill, termed as the normreducing technique, has been proposed by [25][26][27].Moreover, following the idea presented by [26], it is easy to find that when  is real,  ∞ can be chosen to be a real normal matrix.In a word, any real matrix  can be translated into a real normal matrix  ∞ =  −1 ∞  ∞ by a similarity transformation  ∞ =  1  2 ⋅ ⋅ ⋅.Typical approaches for constructing   can be found in [26].Note that if  is the eigenvalue of  ∞ corresponding to the eigenvector ,  is the eigenvalue of  corresponding to the eigenvector  ∞ .Hence, our proposed algorithm can be extended to extract eigenpairs of arbitrary real matrices by employing the norm-reducing technique.
In the following, we use  = 1000 random matrices to verify the average performance of the norm-reducing technique.Let denotes the average measure of a large number of nonnormal matrices in a statistical sense at th iteration, where 2 is the measure of the nonnormal matrix   at th iteration, Δ 2  = 0 if and only if   is normal matrix at th iteration.We also presented the dynamic behavior trajectory of Mean Δ 2 in Figure 2, from which we can see that for most of nonnormal matrices, after 350 iterations, the Mean Δ 2 is very close to zero.

Neural Implementation Description
In the presented paper, we mainly focus on the classical neural network differential equation as shown in (2), where  = (  ), ,  = 1, 2, . . .,  are symmetric matrices that need to calculate eigenvalues and the corresponding eigenvectors, () = [ 1 (),  2 (), . . .,   ()]  is a column vector which denotes the states of neurons in the neural network dynamic system, and the elements of symmetric matrix  denote the connection weights between those neurons.We presented the schematic diagram of the neural network in Figure 3, from which we can see that it is a recurrent neural network since the input is just the output of the system.In the practical applications, we often only need a nonzero column vector (0) = [ 1 (0),  2 (0), . . .,   (0)]  to start the neural network system by the following update rule:

Examples and Discussion
Three experiments are presented to verify our results.The following real normal matrix  (randomly generated) was used in those three experiments: ) . ( We can see that all of (C1) to (C6) hold except (C2).For simplicity, denote lim  → ∞ () by (∞).
Although we can use (25) to get the smallest absolute value of the imaginary part among   (it is zero in this experiment), neither the corresponding real part nor the eigenvector can be obtained from Lemmas 8 or 9 since (C2) does not hold.in Figures 8 and 9.After convergence, we saw (57) Hence, the estimated complex vector is an eigenvector of  corresponding to  5 .
Based on (37), we got  1 (the smallest real part among   ).After convergence, we saw that (∞)    ( According to the results above, the eigenvalues with largest imaginary in absolute are  2 and  3 , and let  2 ± | 2 | denote them. We used (10) with the following initial condition (randomly generalized): from which we can see that the estimated complex vector is an eigenvector of  corresponding to  2 .

Conclusion
This paper introduces a neural network based approach for computing eigenvectors of real normal matrices and the corresponding eigenvalues that have the largest or smallest modulus, have the largest or smallest real part, and have the largest or smallest imaginary part in absolute value.All the computation can be carried out in real vector space although eigenpairs may be complex, which can reduce the scale of networks a lot.We also shed light on extending this method to the case of general real matrices by employing the norm-reducing technique proposed in other literatures.Four simulation examples verified the validity of our proposed algorithm.

Figure 3 : 4 =
Figure 3: The schematic diagram of the neural network equation (2) for solving the eigenvector corresponding to the modulus largest eigenvalues of real symmetric matrix .

Example 20 (Figure 5 :Figure 6 :
Figure 5: Trajectories of (), the solution of (10) with initial (0) as (55), which should converge to the imaginary part of an eigenvector of  corresponding to the eigenvalue  3 .
Arbitrary Real Matrices.In this subsection,  is an arbitrary real matrix of order .Let ‖‖ be the Frobenius norm of , and,   ,  = 1, . . ., , be the eigenvalue of .Denote the set of all complex nonsingular matrices by T.