FromPenroseEquations toZhangNeuralNetwork,Getz–Marsden Dynamic System, and DDD (Direct Derivative Dynamics) Using Substitution Technique

School of Computational Science, Zhongkai University of Agriculture and Engineering, Guangzhou 510220, China School of Computer Science and Engineering, Sun Yat-Sen University, Guangzhou 510006, China Research Institute of Sun Yat-sen University in Shenzhen, Shenzhen 518057, China Guangdong Key Laboratory of Modern Control Technology, Guangzhou 510070, China Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, Guangzhou 510006, China


Introduction
During the past decades, scientists and engineers have encountered linear matrix equation problems, i.e., Lyapunov equation, Sylvester equation, and the variational problem, again and again in various scenarios. e matrix inversion problem is one of the most prominent subproblems in the linear matrix equation problems. Among the problems encountered in a variety of optimization problems, the fundamental one is the solution of matrix inversion, such as signal processing [1], biomedical prediction [2], image reconstruction [3], nonlinear optimization [4], and robot inverse kinematics [5][6][7][8]. Generally speaking, the matrix inversion problem can be formulated as AX � I, where A ∈ R n×n is a known constant matrix, X ∈ R n×n is the unknown matrix to be computed, and I ∈ R n×n is an identity matrix. Over the years, efforts were directed towards computational issues of time-variant matrix inversion (TVMI) and a wealth of algorithms were proposed and applied to solving this problem [9][10][11]. For example, Yeung and Kumbi [9] developed an inversion method based on the multidimensional discrete Fourier transform of matrix sequences and applied it to an electron amplifier. In [10], Benner et al. adopted Gauss-Huard algorithm to solve TVMI problem. In [11], Xiao et al. designed a complexvalued nonlinear recurrent neural network to solve timevarying complex matrix inversion. Zhang et al. [12] discussed the solution of TVMI problem based on direct derivative dynamics (DDD) and proposed a zero-stable, consistent, and convergent four-step model.
According to the classic solution of matrix inversion, the time complexity of arithmetic operations is proportional to O(n 3 ), where n represents the dimension of the square matrix [13]. Evidently, it is an unbearable cost for solving the TVMI problem. erefore, it is an urgent need for effective methods in TVMI problem solving.
In recent years, with the rise of the artificial neural network, the recurrent neural network (RNN) has been thought and manifested to be an effective option for TVMI problem solving. Methods based on RNN have been proved to be promising for RNN's nature of high-speed parallel-processing and convenience of hardware implementation [14][15][16][17][18]. To solve the TVMI problem, Zhang et al. firstly proposed a new class of RNN, Zhang neural network, abbreviated as ZNN [19]. e essence of ZNN model is to construct a carefully selected error-monitoring function, termed Zhang function (ZF). e ZF can be negative, zero, positive, bounded, or even lower-unbounded. e traditional error-monitoring functions are often some norm-based positive-definite energy functions, which are not so flexible as ZFs. Moreover, since the time-derivative information of the time-variant matrix is fully utilized by ZFs, the resultant ZNN model can decrease the lagging errors which are almost unavoidable in the traditional methods. In theoretical and practical researches, the ZNN models claim their natural advantages in convergence and high accuracy. Inspired by this idea, the ZFs have been found to speed up and consolidate the development of various ZNN models [20][21][22][23][24][25][26].
With the deepening of the research on solving TVMI, designing a new ZF through specific classic equations, deriving new solution models, and discussing the convergence and accuracy of the models have become the key to the solution of the extended TVMI problem.
As a generalization of matrix inverse, the Moore-Penrose generalized inverse has been widely investigated [27]. It is applied to finding the least norm square solution of nonuniform linear equations and makes the form of solution simple. e Moore-Penrose generalized inverse of a matrix is unique in both real and complex fields, which can be obtained theoretically by singular value decomposition (SVD) algorithm [27]. Although the SVD algorithm provides a direction for TVMI problem solving, it is deeply troubled by the high time complexity of the algorithm. For example, according to the actual number of multiplication operations, SVD algorithm needs 2mn 2 + 4n 3 operations to find the Moore-Penrose generalized inverse of an m × n matrix [27]. erefore, based on the mathematical definition of inverse matrix, there have been many attempts to solve the TVMI problem [28][29][30].
In view of the fundamental theoretical value of Penrose equations (PEs) in the definition of generalized inverse, it is necessary to study the novel TVMI solution models and find equivalence between existing solution models from PEs. But as far as we know, there is no relevant research so far. is paper focuses on the PE relevant models. By constructing different matrix error-monitoring functions from PEs, three corresponding time-variant matrix inverse solution models are derived. rough theoretical derivation and computer simulation, the feasibility and the effectiveness of the new models in TVMI problem solving are verified. e remainder of this paper is organized as follows. Section 2 explains the PEs and some necessary equations. In Section 3, we derive a new ZNN multiple-multiplication model for matrix inversion, abbreviated as ZMMMI, from PEs. ereafter, we employ the Euler forward finite difference (two-instant) formula, the four-instant Taylor-Zhang formula, the six-instant Zhang time discretization (ZTD) formula [31][32][33], the eight-instant ZTD formula, and the teninstant ZTD formula to develop five discrete algorithms to compute TVMI. In Section 4, we derive a new PEs-based Getz-Marsden dynamic system (PGMDS) model for timevariant matrix inversion from PEs. ereafter, we employ the five discrete formulas to present five discrete algorithms to compute TVMI. In Section 5, we present a DDD (direct derivative dynamics) neural network model for time-variant matrix inversion on the basis of PEs. ereafter, we employ the five discrete formulas to develop five discrete algorithms to compute TVMI. In Section 6, we conduct numerical experiments to verify the convergence and precision of the three new models. Based on the experiment results, we compare and discuss the effectiveness of the three models. Section 7 gives the conclusions and future directions of research.
rough the research of this paper, more TVMI solution models will be available to researchers. e main contributions of this paper can be outlined as follows.
(1) ree different models for solving the TVMI are proposed by defining different error-monitoring functions from PEs for the first time. is is the main motivation of this paper. (2) We propose and provide a novel ZMMMI model from PEs, of which the stability and convergence are proved theoretically, thereby being rare complements to the problem solving of TVMI. (3) e paper investigates and provides the theoretical analysis of the continuous-time models and finds that the classic GMDS model can also be derived from the PEs.

Penrose Equations
For the purpose of laying a basis for direction, some necessary preliminaries of the time-variant matrix where t denotes the time, superscript T indicates the matrix transpose operator, and X(t) represents the time-variant pseudoinverse of A(t), which is often denoted by A + (t).
When rank (A(t)) is equal to m and A(t)A T (t) is nonsingular, the unique X(t) can be obtained as Equation (2) holds true when A(t) is full row rank. Similarly, when A(t) is full column rank and A(t)A T (t) is nonsingular, unique X(t) can be obtained as In equations (2) and (3), we use superscript − 1 to indicate the matrix inversion operator. Also, X + (t) is called the right and left pseudoinverse of A(t) in (2) and (3), respectively.

New ZNN Model and Algorithms
is section focuses on constructing another new ZF from PEs and then derives a new continuous-time solution model for Penrose pseudoinverse based on ZNN design formula [35].

Continuous-Time Model from PEs.
e design formula of ZNN is shown below: where c ∈ R + is the parameter whose physical meaning is the reciprocal of the product of the corresponding capacitance parameter and the resistance parameter. According to [28], (4) is a first-order differential equation whose norm of the general solution is inversely proportional to the exponential function of c. erefore, in order to make Z(t) converge to zero as rapid as possible, c should be set as large as the physical hardware allows. Next, we discuss the ZNN model in the real situation. We choose (1a) to be the ZF as and when t ⟶ + ∞, Z(t) ⟶ 0 theoretically. en, we take the derivative of both sides of (5) and get Substituting (5) and (6) into (4), we obtain Reformulating (6), we have In order to derive the multiple-multiplication model, one assumes that A(t) is square and of full rank. To get the explicit _ X(t), we need to left multiply both sides of (8) by A − 1 (t) and right multiply both sides of (8) by A − 1 (t) and then get explicit _ X(t) as e goal of the model design in this section is to use the known matrix to estimate _ A(t), but A − 1 (t) appears on the right-hand side of (9), so it is necessary to use the substitution technique to replace A − 1 (t) with the appropriate known matrix. Based on eorem 1 in [36], the state matrix X(t) of (1a) globally converges to A − 1 (t) when the sampling gap ϵ ⟶ 0 and t evolves large enough.
erefore, X(t) would be a feasible substitute for A − 1 (t). is substitution technique has been proven to be effective in [37][38][39]. erefore, we have the explicit dynamics of the ZNN models as is novel model differs from any previous TVMI solution model. Note that the right-hand side of (10) involves multiple matrix-multiplication, so the model is termed as continuous-time ZMMMI model (10). To prepare for the following discussion, a lemma is given below [28].

Lemma 1. For a time-variant real matrix A(t) ∈ R n×n and its inverse
Proof. Because A(t)A − 1 (t) � I with I ∈ R n×n being the identity matrix, we have Expanding the left-hand side of the above equation, we obtain Discrete Dynamics in Nature and Society Next, left multiply A − 1 (t) on both sides of the above equation and then reformulate it. us, we obtain For simplicity, the notation d(·)/dt is substituted by (·) . . en, we obtain e proof is thus complete.

□
For the continuous-time ZMMMI model (10), we provide the following proposition on its exponential convergence performance with a proper random initial state.
We get the analytic solution of (18) in the following matrix form: en, we further obtain where symbol ‖ · ‖ F denotes the Frobenius norm of a matrix and e represents Euler number. According to (21), as t ⟶ ∞, ‖W(t)‖ F ⟶ 0 with rate c > 0 exponentially. e proof of exponential convergence of ZMMMI model (10) is thus completed.

Discrete
Algorithms. e hardware implementation of the continuous-time ZMMMI model (10) needs discrete algorithm. In this section, we discuss five discrete algorithms for the continuous-time ZMMMI model (10).

Euler Forward Formula-Based Algorithm.
For simplicity, the following Euler forward difference method is referred [43,44]: where ϵ > 0 represents the sampling gap and k � 0, 1, 2, . . . represents the iteration number. t k � kϵ denotes the sampling time at iteration number k. For simplicity, we denote X k � X(t k ), and then (22) is transformed as With (23), we discretize the continuous-time ZMMMI model (10) as 4 Discrete Dynamics in Nature and Society which is further formulated as where h � ϵc > 0 denotes sufficiently small step length. Because it only depends on two sampling points in the past time, algorithm (25) is named as ZMMMI2i algorithm. It is worth pointing out that ϵ > 0 should be set sufficiently small to ensure the convergence of the discrete model of ZMMMI model (10).

Taylor-Zhang Discretization
Formula-Based Algorithm. According to [45], TZDF is presented as With (10) and (26), the four-instant ZMMMI algorithm is named as ZMMMI4i algorithm, which is expressed as follows: where h > 0 denotes the step size again as before.

Six-Instant ZTD Formula-Based Algorithm.
In [37], a six-instant ZTD formula is given as follows: With (10) and (28), the six-instant ZMMMI algorithm is named as ZMMMI6i algorithm, which is given as follows: where h > 0 denotes the step size again as before.

Eight-Instant ZTD Formula-Based Algorithm.
In [37], an eight-instant ZTD formula is given as follows: With (10) and (30), the eight-instant ZMMMI algorithm is named as ZMMMI8i algorithm, which is given as follows: Discrete Dynamics in Nature and Society
When h is not fixed and c is a constant, the error is upgraded to O(ϵ 5 ) as a result.

Theorem 1. Consider a smoothly time-variant real matrix
Proof. According to eorems 1 and 2 in [46], we know the fact that (25) is 0-stable, consistent, and convergent. As a result, it converges with the order of its truncation error. Assume that B k+1 is the exact solution of A k+1 B k+1 � I. According to (25), (27), (29), (31), and (33), we have e proof is thus completed.

GMDS Model and Algorithms
is section focuses on constructing new ZF from PEs and then derives continuous-time PGMDS solution model for matrix inverse based on ZNN design formula. Furthermore, we develop five discrete algorithms for continuous-time PGMDS model by exploiting the multiple-instant ZTD formulas.

Continuous-Time Model from PEs.
In this section, we choose (1b) to be the ZF as and when t ⟶ + ∞, Z(t) ⟶ 0 theoretically. en, we take the derivative of both sides of (1b) and get Substituting (35) and (36) into (4), we obtain According to the fact that A(t)X(t) � X(t)A(t) � I, we reformulate (36) and then obtain Model (38), which is derived from PE (1b), is exactly the Getz-Marsden dynamic system mentioned in [36]. erefore, the model is named as continuous-time PGMDS model (38).

Discrete Algorithms.
In this part, we discretize the continuous-time PGMDS model (38) with the following five discrete formulas as those in Section 3.

Euler Forward
With (39) and (23), the two-instant PGMDS algorithm is named as PGMDS2i algorithm, which is expressed as follows: where h � ϵc > 0 denotes the step length as before.

Taylor-Zhang Discretization
Formula-Based Algorithm. With (39) and (26), the four-instant PGMDS algorithm is named as PGMDS4i algorithm, which is expressed as follows: where h > 0 denotes the step size again as before.

Six-Instant ZTD Formula-Based Algorithm.
With (39) and (28), the six-instant PGMDS algorithm is named as PGMDS6i algorithm, which is given as follows: where h > 0 denotes the step size again as before.

Eight-Instant ZTD Formula-Based Algorithm.
With (39) and (30), the eight-instant PGMDS algorithm is named as PGMDS8i algorithm, which is given as follows: where h � ϵc > 0 denotes the step length as before.

Ten-Instant ZTD Formula-Based Algorithm.
With (39) and (32), the ten-instant PGMDS algorithm is named as PGMDS10i algorithm, which is thus obtained: where h � ϵc > 0 denotes the step length as before.

Steady-State Residual Errors of Discrete Algorithms.
Next, a theorem is given to show that the steady-state residual of lim k⟶+∞ sup‖X k+1 A k+1 X k+1 − X k+1 ‖ F is equivalent to the precision of the corresponding discrete algorithm in this section.
Proof. With the same conditions as eorem 1, we know the fact that (39) is 0-stable, consistent, and convergent [46]. As a result, it converges with the order of its truncation error. Assume that B k+1 satisfies A k+1 B k+1 � I and X k+1 � B k+1 + O(ϵ p ) with ϵ ∈ (0, 1) as before. We further have e proof is thus completed.

DDD Model and Algorithms
is section focuses on constructing direct derivative dynamics from one appropriate Penrose equation and then derives direct derivative solution model for matrix inversion.

Continuous-Time Model from PEs.
We take the derivative on both sides of (1a) and then obtain _

A(t)X(t)A(t) + A(t) _ X(t)A(t) + A(t)X(t) _
To get the explicit expression of _ X(t), we reformulate above equation and obtain Discrete Dynamics in Nature and Society According to the assumption of the matrix inverse problem, we can substitute A − 1 (t) with X(t) in (47) and thus obtain _

X(t) � −X(t) _ A(t)X(t) − X(t) _ A(t)X(t) + X(t) _ A(t)X(t).
(48) e right-hand side of above equation can be simplified as Finally, we present the continuous-time direct derivative dynamics model (49), which is termed as continuous-time DDD model (49) for short.

Discrete Algorithms.
In this part, we discretize the continuous-time DDD model (49) with the following five discrete formulas as those in Section 3.

Euler Forward Formula-Based Algorithm. First, (49) is discretized as
With (50) and (23), the two-instant DDD algorithm is named as DDD2i algorithm, which is expressed as follows:

Taylor-Zhang Discretization
Formula-Based Algorithm. With (50) and (26), the four-instant DDD algorithm is named as DDD4i algorithm, which is expressed as follows:

Six-Instant ZTD Formula-Based Algorithm.
With (50) and (28), the six-instant DDD algorithm is named as DDD6i algorithm, which is given as follows:

Ten-Instant ZTD Formula-Based Algorithm.
With (50) and (32), the ten-instant DDD algorithm is named as DDD10i algorithm, which is thus obtained: Discrete Dynamics in Nature and Society

Computer Experiments and Results
In this section, computer experiments is carried out to verify the effectiveness of the presented three models, ZMMMI model (10), PGMDS model (38), and DDD model (49), on three time-variant matrix inversion examples.

Example 1.
Let us consider the following discrete-time matrix inversion problem with X k+1 to be obtained during [t k , t k+1 ), of which A k is defined as e task duration (i.e., final time) is uniformly set as t d � 30 s. To verify the computational results, the theoretical inversion of matrix (56) can be obtained as for all algorithms.  Figures 1 and 2. Figure 1(a) shows the residual errors ‖X k+1 A k+1 − I‖ F of the five ZMMMI algorithms, with ϵ � 0.001 s and h � 0.005. Figure 1(b) illustrates the residual errors ‖X k+1 − A −1 k+1 ‖ F of the same five ZMMMI algorithms as before. From the trajectories of all entries in Figures 2(a)-2(d), the solution of the model coincides with the theoretical solution perfectly. In addition, we see that the residual error trajectories synthesized by different ZMMMI algorithms quickly stabilized to the steady-state error level, after undergoing the initial hundreds of recursions.  Figure 5(a) shows the residual errors ‖X k+1 A k+1 − I‖ F of the five DDD algorithms, with ϵ � 0.001 s and h � 0.005. Figure 5(b) illustrates the residual errors ‖X k+1 − A −1 k+1 ‖ F of the same five DDD algorithms as before. From the trajectories of all entries in Figures 6(a) and 6(b), even for the model with the highest accuracy, DDD10i algorithm (55), its solution still cannot converge to the theoretical solution.

Example 2.
e second time-variant matrix is a 3 × 3 real matrix which is shown as follows: Discrete Dynamics in Nature and Society where A(t k ) ∈ R 3×3 . To verify the computational results, we utilize the theoretical inverse of A(t k ), which is denoted as

PGMDS Model.
e results of the numerical experiments for PGMDS2i algorithm (40), PGMDS4i      convergence and accuracy, the experimental results in this example are consistent with those in example 1.

Example 3.
is example compares solely the effective models, i.e., ZMMMI (10) and PGMDS (38), with a 4 × 4 real matrix, which is shown as follows: where A(t k ) ∈ R 4×4 and s k and c k denote sin(t k ) and cos(t k ), respectively. To verify the computational results, we utilize the theoretical inverse of A(t k ), which is denoted as e task duration is uniformly set as t d � 30 s.
e experimental results are satisfactory as expected.

Comparison and Discussion.
We mainly investigate the residual error of the models and the coincidence between the solution matrix with the ground-truth matrix inverse. From the discrete simulation results of two examples of each five discrete algorithms, it can be seen that the convergence of ZMMMI model (10) and PGMDS model (38) is good, which is completely consistent with the conclusion of Proposition 1 and eorems 1 and 2. erefore, it can be concluded that the two models and corresponding discrete algorithms are effective. However, in the case of DDD model (49), the residual errors and the coincidence of the solution matrix entries are not satisfactory, so we evidently summarize that the effectiveness of DDD model (49) and corresponding discrete algorithms is not enough.

Remark.
Note that the convergence of DDD models is shown in [12,47], where the DDD models are utilized to solve the time-varying nonlinear optimization problems.
ere are three points worth further discussing. First, in [12], the initial value is set to be the theoretical value, and the   computational process is not disturbed. Second, comparatively, in this study, the initial value is not close enough to the theoretical value. ird, the higher-order discrete algorithm (i.e., the ten-instant discrete algorithm) is more easily affected by the rounding error disturbance [44]. us, the experimental results of the DDD model in this study showing the divergence are actually complementing the previous research studies, in addition to the confirmation of [47] about divergence. erefore, we summarize that the DDD model is generally less effective, with further in-depth investigation being also a future research direction.

Conclusion
In this paper, we have shed some light on the matrix inversion solution models derivation, i.e., ZMMMI model (10), PGMDS model (38), and DDD model (49), from PEs. It has provided a new perspective to make full use of the theoretical value of PEs. First, with the substitution technique and design formula of ZNN, we have investigated and proposed the new model of ZNN for matrix inversion problem. en, we have discussed the convergence and accuracy of ZMMMI model (10) and presented two theorems and proofs about it. On the basis of the model, we have developed five ZMMMI algorithms to discretize continuoustime ZMMMI model (10). Second, with the substitution technique and design formula of ZNN, we have investigated and presented PGMDS model (38) for matrix inversion problem. On the basis of the model, we have shown five PGMDS algorithms to discretize continuous-time PGMDS model (38).
ird, with the substitution technique and design formula of ZNN, we have presented DDD model (49) for matrix inversion problem. On the basis of the model, we have developed five DDD algorithms to discretize continuous-time DDD model (38). Fourth, we have prepared three examples to calculate the inverse of matrices by using the above three models, respectively. e results illustrate that ZMMMI model (10) and PGMDS model (38) are effective, while DDD model (49) is less effective. In the future, based on PEs, we will use the substitution technique and ZNN design formula to discuss more efficient RNN models for TVMI problem, including pseudo-inversion and complex matrix inversion.

Data Availability
No data were used to support this study.

Conflicts of Interest
e authors declare that they have no conflicts of interest.