New Results on Passivity Analysis of Stochastic Neural Networks with Time-Varying Delay and Leakage Delay

The passivity problem for a class of stochastic neural networks systems (SNNs) with varying delay and leakage delay has been further studied in this paper. By constructing a more effective Lyapunov functional, employing the free-weighting matrix approach, and combining with integral inequality technic and stochastic analysis theory, the delay-dependent conditions have been proposed such that SNNs are asymptotically stable with guaranteed performance. The time-varying delay is divided into several subintervals and two adjustable parameters are introduced; more information about time delay is utilised and less conservative results have been obtained. Examples are provided to illustrate the less conservatism of the proposed method and simulations are given to show the impact of leakage delay on stability of SNNs.


Introduction
During the past several decades, neural networks have gained great attention because of their potential application in pattern classification, reconstruction of moving image, and combinatorial optimization. In addition, time delay is a natural phenomenon frequently encountered in various dynamic systems such as electronic, chemical systems, long transmission lines in pneumatic systems, biological systems, and economic and rolling mill systems. Delays in neural networks can cause oscillation, instability, and divergence, which are very often the main sources of poor performance of designed neural networks. So the stability analysis and state estimation of neural networks with various time delays have been widely investigated by many researchers; see [1][2][3][4][5][6][7][8] and the references therein.
Furthermore, when modeling real nervous systems, stochastic disturbance is one of main resources of the performance degradations when applying the neural networks, because the synaptic transmission is a noisy process introduced by random fluctuation from the release of neurotransmitter and other probabilistic causes. In recent years, the stability analysis for stochastic neural networks with time delay has become a hot research topic; by virtue of various inequality technics and -matrix theory, many important research results about neural networks with different type of time delays, such as constant delay, time-varying delay, or distributed delay, have been reported; see, for example, [8][9][10][11][12][13][14] and the references therein.
The passivity theory, which originated from circuit theory, plays an important role in the analysis of stability of linear or nonlinear systems. The main character of passivity theory is that the passive properties of a system can keep the system internally stable. Because it is a very effective tool in studying the stability of uncertain or nonlinear systems, the passivity theory has been used widely in fuzzy control [15], complexity [16], synchronization [17], signal processing [18], and adaptive control [19].
In [29][30][31][32][33][34][35], based on the Lyapunov-Krasovskii, LMI method, and a delay fractioning technique, the passivity and robust passivity of stochastic neural networks with delays and uncertainties have been studied; some sufficient conditions on the passivity of neural networks with delays have been obtained. In [31], authors investigated passivity of the stochastic neural networks with time-varying delays and parameters uncertainties by applying free-weighting matrix and the 2 Computational Intelligence and Neuroscience lower conservatism results are obtained by comparing with the existing results.
On the other hand, in many practical problems, a typical time delay called leakage delay or forgetting delay exists in dynamical system, which has a tendency to destabilize the system; it has been one of the research hot topics recently and many research achievements have been reported [20,[36][37][38][39][40][41][42].
As pointed out in [36], neural networks with leakage delay are a class of important neural networks, and time delay in the leakage term also has great impact on the dynamics of neural networks; sometimes it has more significant effect than other kinds of delays on dynamics of neural networks; the stability analysis of neural networks system involving leakage delay has been researched extensively; see, for example, [37][38][39][40] and the references therein. Very recently, in [42], by virtue of free weight matrix and LMIs method, the passivity problem for a class of stochastic neural networks with leakage delay is studied; the sufficient condition making the system passive is presented, but leakage delay under consideration is a constant; but, in practical dynamical systems, the leakage delay can be time-varying, which is often more general and complex than leakage delay being a constant. To the best of authors' knowledge, no research results have been reported about the condition that leakage delay is time-varying, which motivates our idea.
Motivated by the aforementioned discussions, this paper focuses on the passivity problem for a class of stochastic neural networks (SNNs) system with time-varying delay and leakage delay; by constructing a new Lyapunov functional, a set of sufficient conditions are derived to ensure the passivity performance for a class of stochastic neural networks with time-varying delays and leakage delay. By virtue of the delay decomposition idea [8], combining with some integral inequality technic [7], or free-weighting matrix approach [9,26], two adjustable parameters are introduced and made full use of. All results are established in the form of LMIs and can be solved easily by using the interior algorithms, which can be efficiently solved by Matlab LMI Toolbox and no tuning of parameters is required. Finally, numerical examples are given to demonstrate the effectiveness and less conservatism of the proposed approach.
The main contributions of this paper are summarized as follows: (i) The leakage delay studied is time-varying, so the research model is more general and complex than that in [42].
(ii) The neuron activation function is assumed to satisfy sector-bounded condition, which is more general and less restrictive than Lipschitz condition, so the less conservatism results can be expected.
(iii) The derivative of time-varying can be extended to be more than 1.
(iv) How the leakage delay affects the stability result is discussed.
Notation. Throughout this paper, if not explicit, matrices are assumed to have compatible dimensions. The notation > (≥, <, ≤) 0 means that the symmetric matrix is positive-definite (positive-semidefinite, negative, and negative-semidefinite). min (⋅) and max (⋅) denote the minimum and the maximum eigenvalue of the corresponding matrix; the superscript " " stands for the transpose of a matrix; the shorthand diag{⋅ ⋅ ⋅ } denotes the block diagonal matrix; ‖ ⋅ ‖ represents the Euclidean norm for vector or the spectral norm of matrices. refers to an identity matrix of appropriate dimensions. E{⋅} stands for the mathematical expectation; * means the symmetric terms. Sometimes, the arguments of a function will be omitted in the analysis when no confusion can arise.

System Description
Consider the SNNs with time-varying delay as follows: (2) ( ) is the transmission delay and is assumed to satisfy ( ) is the leakage delay that satisfies where , , , are some positive scalar constants.
where Λ 1 and Λ 2 are some constant known matrices.
Computational Intelligence and Neuroscience 3 Remark 2. In this paper, the above assumption is made on neuron activation function, which is called sector-bounded neuron activation function. When Λ 1 = Λ 2 = −Λ, condition (5) becomes So it is less restrictive than the descriptions on both the sigmoid activation functions and the Lipschitz-type activation functions.
Assumption 3. There exist three constant matrices Σ 1 , Σ 2 , and Σ 3 such that Definition 4 (see [22]). The delayed SNNs are said to stochastically passive if there exists a scalar ≥ 0 such that for all ≥ 0 and for all solution of (1) with (0) = 0.
Remark 5. The different output equation can lead to different definitions. In [31,42], the output equation expression is ( ) = ( ( )) and ( ) = ( ( )), respectively. In order to compare our result with that in [42], we take = , so we have the same definition as that in [42].
At first, we give the following lemmas which will be used frequently in the proof of the our main results.

Main Results
In this section, a delay-dependent leakage delay method is developed to guarantee the stochastic passive results of system (1), so we have the following Theorem 10.
It is clear that for any scalars 1 > 0 and 2 > 0, there exist diagonal matrices 1 ≥ 0, 2 ≥ 0, and Λ ( = 1, 2) such that the following inequality hold: In order to get the passive condition, we introduce the following inequality: On the other hand, for formulas (25)-(28), we further have At the same time, from the character of Itô integrals, we can obtain that Computational Intelligence and Neuroscience By substituting (22)-(23) into (20) and considering (36), then taking expectation on both sides of (20), and then using (38), we can get By Lemma 8, there exist nonnegative functions 1 ( ) and 2 ( ) satisfying 1 ( ) + 2 ( ) = 1 such that Substituting (43) into (42), then (42) can be rewritten as So we can get that the following matrix inequalities hold: By virtue of Lemma 7,(45) and (46) are equivalent to (13) and (14), respectively, so we can get that (47) then integrating on both sides of (47) from 0 to , we can obtain It indicates that system (1) is stochastically passive in the sense of Definition 4. This completes the proof.
This system has been studied in [42]; then for system (49) we have the following Corollary 12.
It is well known that the Markovian jump systems (MJSs) are a special class of hybrid systems, which have the advantage in modeling the dynamic systems subject to abrupt variation in their structures, such as component failures and sudden environmental disturbance. Many researches about the stability analysis, impulsive response, and state estimation on the neural networks with Markovian jumping parameters have been obtained; see [44][45][46][47] and references therein. Recently [48] has studied the passivity of stochastic neural networks with Markovian jumping parameters; the same method can be used to a system with Markovian jumping parameters and it still leaves much room to reduce the conservatism, which motivates our aim.
Let , ≥ 0, be a right-continuous Markov chain defined on a complete probability space (Ω, F, ) and taking discrete values in a finite state space = {1, 2, . . . , } with generator Π = ( ) × given by where Δ > 0 and ≥ 0 is the transition rate from to while For the purpose of simplicity, in the sequel, for each = ∈ , ( ), 0 ( ), and 1 ( ) are denoted by , 0 , 1 , and so on. Throughout the paper, we assume that ( ) and (55) This system has been studied in [43] and good results have been obtained. In order to testify the effectiveness of our methods, we give the following Theorem 13.
By the same method as that in Theorem 10, we can get that the following inequalities hold: So by virtue of Lemma 7 and the same proof method of Theorem 10, we can get that system (55) is stochastic passive.

Numerical Example and Simulation
In this section, three numerical examples are presented to demonstrate the effectiveness of the developed method on the obtained passive results.
Take the activation function as 1 ( ( )) = 2 ( ( )) = tanh( ), so it can be verified from Assumption 1 that 1 = diag{0, 0} and 2 = diag{−0.5, −0.5}, and by using of the Matlab LMI Control Toolbox, we find out a solution to LMIs (12), (13), and (14) as follows: In order to testify the effectiveness of our proposed method, many experiments have been done and the upper bounds of delays and are listed from Tables 1 to 3, where "-" means that LMIs (12)- (14) has no feasible solution. Table 1 shows the maximum allowable upper bound for different values of , which means that the bound of the derivative of the leakagetime-varying is very effective and plays an important role in obtaining the feasible results. From Table 2, we can see that when > 1, the feasible solution can be obtained. From Table 3 we can see that when fixing the value of and , the allowable upper value of is effected by , especially when = 0.45; the feasible solution cannot be obtained. Especially, when leakage ( ) = , namely, leakage delay, is constant, the studied system will become system (40), which has been researched in [42]; then we have the following Example 2.
Example 2. Consider that stochastic neural networks (49) have the same parameters as that in [42], so from Corollary 12, we can have the following research results listed in Tables 4  and 5, which show the effect on for different and mutual effect between and .
From Table 4 we also can see that when the same parameters in (2) and (3) of [42] are taken into account and  when is set by 1 and = 0.1 and 1 = 2 = 0.1 * , by solving (2) and (3) of [42], we can get that maximum value is 0.0005, so our method has obtained the less conservatism than that of [42]. In this example, when ( ) = [−0.3 cos(3.1 ) 0.7 sin(1.4 )] , Figure 1 shows the state curve with ( ). From Figure 1 we can see that when stochastic disturbance and input exist, the systems with leakage delay are unstable.

Remark 14.
In [42], the sufficient conditions of passivity about stochastic neural networks are given by LMIs, but the solution is not given out, and the simulation about both stochastic and leakage delay is not discussed, either. In our discussion, the impact of leakage delay on stability of systems is considered.
At the same time, when leakage delay is set by different values, by taking the initial state [2, −1] and using the Matlab software, a state curve is obtained as in Figure 2; Figure 2 shows the state curves of system (49) without input and is 0.6 and = 0.8; Figure 3 shows the state curves of system (49) without input and is 0.2 and = 0.8.
When stochastic disturbance does not exit in system (49), the state simulation curves of (49) are shown in Figure 4. Figure 4 shows the state curves of system (49) without stochastic disturbance and different leakage delay; from Figure 4, we can see that when the leakage delay exits in the neural networks system, state curve of system oscillates sharply from the start point and then becomes asymptotically stable; at the same time, we can find that the bigger the leakage delay, the more serious the oscillation.

Remark 15.
In Corollary 2 of [42], the maximum value of time delay is 0.2, when leakage delay is set by 0.1. In our method, combing the simulation curve with the value of , when is 0.1, the maximum value of time delay which is guaranteeing the fact that system (1) is stable can reach 1.2.
Remark 16. In [42], an example has been given to show the effectiveness of passivity criteria, but how the leakage delay affects the stability is not discussed. In our example, simulations have been given and proved that leakage delay can cause effect on the stability of neural networks.
At the same time, in order to testify the less conservatism of our method, the allowable upper bounds of with different values of have been compared with that in [43]; the results are shown in Table 6. From Table 6, we can see that even > 1; our methods can improve existing research results.
On the other hand, we select (0) = [0.6, −0.4] and ( ) = [sin( ), * cos( )] , and the following simulation results can be obtained. Figure 5 shows the state curve of system (56) with ( ), Figure 6 shows the state curve of system (56) without ( ), and Figure 7 shows the state switching modes of system (56), so the simulation results further prove that the two-neuron stochastic neural networks with Markovian switching parameters is passive in the sense of Definition 4.

Conclusions
In this paper, we have investigated the passivity problem for a class of stochastic neural networks systems (SNNs) with varying delay and leakage delay. By constructing a novel Lyapunov functional and utilizing the delay fractionizing technique, new passivity conditions have been established to achieve the passivity performance. Moreover, in derivation of the passivity criteria, it is assumed that the description of the activation functions is more general than the commonly used Lipschitz conditions; the time-varying delay is