Delay-Dependent State Estimation of Static Neural Networks with Time-Varying and Distributed Delays

This paper focuses on studying the state estimation problem of static neural networks with time-varying and distributed delays. By constructing a suitable Lyapunov functional and employing two integral inequalities, a sufficient condition is obtained under which the estimation error system is globally asymptotically stable. It can be seen that this condition is dependent on the two kinds of time delays. To reduce the conservatism of the derived result, Wirtinger inequality is employed to handle a cross term in the time-derivative of Lyapunov functional. It is further shown that the design of the gain matrix of state estimator is transformed to finding a feasible solution of a linear matrix inequality, which is efficiently facilitated by available algorithms. A numerical example is explored to demonstrate the effectiveness of the developed result.


Introduction
As a kind of recurrent neural networks [1], static neural networks have attracted more and more attention from the communities of artificial intelligence, nonlinear science, and systems and control during the past few years.Many exciting applications have been established in various areas including combinational optimization, image processing, pattern recognition, knowledge engineering, and semantic web [2].As a matter of fact, time delay is frequently encountered in neural network models and leads to unexpected dynamical behaviors.For example, the existence of time delay may make the underlying neural network unstable or even chaotic (especially when the size of time delay is a bit large).On the other hand, one of the requirements for successful applications is closely related to the stability of the constructed neural network.Consequently, stability analysis of delayed static neural networks has been extensively discussed and some interesting stability criteria have been reported in the open literature (see, e.g., [3][4][5][6][7] and references therein).
With the rapid development of modern industry, the problems to be tackled are of high nonlinearity.Therefore, the neural network applied to solve a complex nonlinear problem often has a great number of neurons.As suggested in [8], it is very hard or expensive to acquire the complete information of the states of all neurons in such a relatively large-scale recurrent neural network, while, in practice, it is necessary to know these types of information in advance and then make use of them to achieve specific objectives [9,10].It is thus of great significance to investigate the state estimation problem of delayed static neural networks.Inspired by [11][12][13], the authors in [14] proposed an improved delay partition approach to dealing with the state estimation problem of static neural networks with time-varying delay.In [15], this issue was studied for discrete-time static neural networks.A delay-range-dependent condition was derived in terms of a linear matrix inequality (LMI).To reduce its conservatism, some free-weighting matrices [16] were introduced.Other related results can be found in [17][18][19].
It should be noted that, in the above mentioned results on state estimation of delayed static neural networks, only timevarying delay was taken into account.In fact, distributed delay, which is distinct from time-varying delay, should be also considered because parallel pathways with various axon sizes and lengths frequently occur in neural networks and the signal transmission between neurons is distributed in 2 Mathematical Problems in Engineering general [20][21][22].That is to say, it is very important to study the effect of distributed delay on state estimation of static neural networks.This is the motivation of the present study.
In this paper, our attention is concerned with the problem of state estimation of delayed static neural networks.Here, both time-varying and distributed delays are considered.To the best of our knowledge, it is the first time to introduce distributed delay in static neural networks.The mathematical model of this kind of delayed static neural networks is presented.Then, by constructing an appropriate Lyapunov functional and employing Jensen and Wirtinger inequalities [23,24], a sufficient condition, which is dependent on both the time-varying delay and distributed delay, is established under which the estimation error system is globally asymptotically stable.Then, the desired gain matrix of state estimator is obtained by solving an LMI [25].An example is finally given to show the effectiveness of the developed result.Although some important results on state estimation of delayed static neural networks were available in [14,15,[17][18][19], the distributed delay has not yet been taken into consideration.One of the contributions of this study is to close this gap and present an efficient approach to handling this issue for delayed static neural networks with the two kinds of time delays.At the same time, some recently proposed techniques are employed to derive a delaydependent criterion such that the implementation of a proper state estimator is easily accomplished based on a convex optimization problem.This is the second contribution of this study.
Notations.Let R be the set of real numbers, R  the dimensional Euclidean space, and R × the set of all  ×  real matrices.For a real matrix ,  > 0 ( < 0) means that  is symmetric and positive definite (negative definite).The superscripts  and −1, respectively, stand for the transpose and inverse of a matrix. is an identity matrix with appropriate dimension.diag{⋅ ⋅ ⋅ } denotes a block diagonal matrix.The symmetric block in a symmetric matrix is denoted by * .Matrices, if not explicitly stated, are assumed to have compatible dimensions.

Problem Formulation and Preliminaries
In the hardware implementations of recurrent neural networks, time delays are unavoidable owing to the finite speed of amplifiers and signal transmission between neurons.In practice, some time delays maybe vary with time.This kind of delays is named as time-varying delays.On the other hand, the so-called distributed delay should be also taken into account since parallel pathways with different axon sizes are actually found and the signal transmission is distributed in a neural network.Therefore, it is reasonable to consider the static neural network with time-varying and distributed delays which is described by where It should be emphasized that the approach which will be developed later can be easily extended to deal with the case that ℎ is also time-varying.Here, just for simplicity, the distributed delay is supposed to be constant. Let with  = 0 or 1.Then, the delayed static neural network (1) can be rewritten as a compact form As discussed in [14], it is difficult or even impossible to fully know the state information of all neurons in a delayed static neural network.However, in some applications, one needs to utilize this information to accomplish desired objectives.In this situation, it is thus very necessary to present an efficient algorithm to implement the state estimation of the underlying static neural network.Then, in place of the "true" states of neurons, the estimated states can be directly used in practice.Generally, one is able to measure the output of a static neural network.As a result, the output measurement plays a key role when estimating the neurons' states.Besides this, the activation function is also fixed in the design process of a neural network.By taking these into account, the output measurement of the delayed static neural network (3) is assumed to be of the form and a state estimator is constructed as ẋ where  is a real known matrix with compatible dimension, (, ()) : R × R  → R  is a nonlinear disturbance on the output measurement, and , to be determined, is a gain matrix of the state estimator.Define the error signal () to be the difference between () and x() (i.e., () = () − x()).It follows from ( 3)-( 5) that the error signal () satisfies Then, (6) can be simplified as Throughout this paper, the following three assumptions are always made.Assumption 2. For  = 1, 2, . . .,  and any different , V ∈ R, the activation function   satisfies where ℓ −  and ℓ +  are constant scalars.
Assumption 3.There exist constant scalars  > 0 and  such that Assumption 4. For any vectors ,  ∈ R  , there are two real known constant matrices Φ 1 and Φ 2 with compatible dimensions such that Remark 5.As mentioned in [21], the scalars ℓ −  and ℓ +  in Assumption 2 can be positive, zero, or negative.It means that the monotonicity is no longer required for the activation function.Therefore, it is more general than the popularly adopted sigmoid functions such as tanh() and Remark 6. Inequality (11) in Assumption 4 is named as the sector-bounded condition [26], which has been widely used in the state estimation theory of delayed neural networks [22,27].
Remark 10.It is known that it is usually difficult to solve a nonlinear matrix inequality.Lemma 9 will be employed to transform a nonlinear matrix inequality into an LMI such that it can be efficiently solved.
Before ending this section, we recall two integral inequalities, which are very essential to the derivation of our main result.
Lemma 11 (Jensen inequality [23]).For any given matrix  ∈ R × with  > 0, scalars ,  ∈ R with  < , and continuous function  : [, ] → R  , then Lemma 12 (Wirtinger inequality [24]).For any given matrix  ∈ R × with  > 0, scalars ,  ∈ R with  < , and continuously differentiable function  : [, ] → R  , one has where Remark 13.It is obvious that, for given  > 0 and Ω, According to Newton-Leibniz formula, when () in Lemma 11 is differentiable, it is known that Wirtinger inequality is less conservative than Jensen inequality.It is thus believed [24] that better performance can be achieved by Wirtinger inequality than by Jensen inequality.In this study, Lemmas 11 and 12 will be, respectively, employed to deal with the terms ∫

Delay-Dependent State Estimation Criterion
Based on Lemmas 7-12, a design criterion is presented for the delayed static neural network (3), which depends on both the time-varying and distributed delays.It is shown that the design of a suitable gain matrix  in ( 5) is transferred to finding a feasible solution of an LMI.Theorem 14.For given scalars , , and ℎ, the resulting error system (8) is globally asymptotically stable if there are real matrices  > 0,   > 0,   > 0, , diagonal matrices Λ  ( = 1, 2, 3), and a scalar  > 0 such that the following LMI is satisfied: where Moreover, the gain matrix  of the state estimator (5) can be designed as Proof.By Lemma 9 and ( 23), one has Pre-and postmultiplying (26), respectively, by diag{, , , , , , , , ,  2  −1 } and its transpose and noting with Mathematical Problems in Engineering According to Schur complement [25], ( 27) is equivalent to where Construct a Lyapunov functional By computing the time-derivative of () along the solutions of the error system (8), one gets where ḋ () ≤  is used to derive the above inequality.By Lemma 7, for diagonal matrices Λ 1 > 0, Λ 2 > 0, and Λ 3 > 0, it is not difficult to deduce that 0 ≤ − 2  ( 0  ()) Λ 1  ( 0  ()) By combining (17) and ( 32)-(34) together, one can deduce Then, it immediately follows from (29) that V() ≤ 0 for any ().According to the theory of Lyapunov stability, the error system ( 8) is globally asymptotically stable.This completes the proof.
Remark 15.The objective of this study is to propose a delaydependent approach to handling the state estimation problem of static neural networks with time-varying and distributed delays.In Theorem 14, a design criterion is derived by means of an LMI.It can be efficiently solved in practice by resorting to some famous algorithms in [25].It should be pointed out that, in order to reduce the conservatism of Theorem 14, Wirtinger inequality is utilized to deal with − ∫  − ė  () 2 ė () in (32), although it can be also estimated by Jensen inequality.

A Numerical Example
Let  = 0.4 and ℎ = 0.5; by solving the LMI (23) in Theorem 14 for different , the gain matrix  can be obtained, which is summarized in Table 1.It is also found that when Now, let  = 0.5 and  = 0.6.We change the value of ℎ and solve the LMI (23).The numerical results are given in Table 2.
For  = 0.5 and  = 0.6, the allowable maximum value of ℎ such that the LMI ( 23

Conclusion
In this paper, the state estimation problem has been studied for a class of delayed static neural networks.Both timevarying and distributed delays have been taken into account.
Based on the Lyapunov stability theory and some integral inequalities, a delay-dependent design criterion has been presented by means of an LMI.So the proper gain matrix can be easily obtained in practice since it is facilitated readily by mature algorithms.Finally, a numerical example has been provided to illustrate the effectiveness of the developed result on the state estimator design of static neural networks with mixed delays.
) is feasible is 1.1539, and the gain matrix  is = 1, 2, . . .,  with  being the number of neurons,   () is the state of the th neuron,   > 0 is a firing rate,   is an activation function of the th neuron,

Table 1 :
The gain matrix  obtained by Theorem 14 for different .