Stability and Bifurcation Analysis of a Three-Dimensional Recurrent Neural Network with Time Delay

We consider the nonlinear dynamical behavior of a three-dimensional recurrent neural network with time delay. By choosing the time delay as a bifurcation parameter, we prove that Hopf bifurcation occurs when the delay passes through a sequence of critical values. Applying the normal form method and center manifold theory, we obtain some local bifurcation results and derive formulas for determining the bifurcation direction and the stability of the bifurcated periodic solution. Some numerical examples are also presented to verify the theoretical analysis.


Introduction
Starting with the work of Hopfield 1 on neural networks, recurrent neural networks including Hopfield neural networks, Cohen-Grossberg neural networks, and cellular neural networks have been used extensively in different areas such as signal processing, pattern recognition, optimization, and associative memories.Many researchers studied the dynamical behavior of Recurrent neural network systems, and most of papers are devoted to the stability of equilibrium, existence and stability of periodic solutions, bifurcation, and chaos 2-5 .In 5 , Ruiz et al. considered a particular configuration of a recurrent neural network, illustrated in Figure 1.In Figure 1, u t is the input and y t is the output of the network.This recurrent neural network can be described by the following system: Here, x t ∈ R n is the state, w i ∈ R, i 1, . . ., n − 1 are the network parameters or weights, u t is a smooth input, and y t is the output.The transfer function of the neurons is taken as f • tanh • .A three-node network of the form 1.1 in feedback configuration, with u t y t , has been studied in 5 ; that is, , where α 2i−1 > 0 for i odd and α 2i−1 < 0 for i even.The authors analyzed the Bogdanov-Takens bifurcation in the system.
It is well known that there exist time delays in the information processing of neurons.The delayed axonal signal transmissions in the neural network models make the dynamical behaviors become more complicated and may destabilize the stable equilibria and admit periodic oscillation, bifurcation, and chaos.Therefore, the delay is an important control parameter in living nervous system: different ranges of delays correspond to different patterns of neural activities see, e.g., 6-11 .In the present paper, we consider the following three-dimensional recurrent neural network model with time delay

1.3
By choosing the time delay as a bifurcation parameter, we prove that Hopf bifurcation occurs in the neuron and study the properties of periodic solutions of this model.The organization of this paper is as follows.In Section 2, by analyzing the characteristic equation of the linearized system of system 1.3 at the equilibrium, we discuss the stability of the equilibrium and the existence of the Hopf bifurcation occurring at the equilibrium.In Section 3, the formulae determining the direction of the Hopf bifurcations and the stability of bifurcating periodic solutions on the center manifold are obtained by using the normal form theory and the center manifold theorem due to Hassard et al. 12 .We do some computer observations to validate our theoretical results in Section 4.  Clearly, x 1 , x 2 , x 3 T 0, 0, 0 T is equilibrium of system 1.3 .Linearization of 1.3 at the zero equilibrium yields The zero equilibrium is stable if all roots of 2.3 have negative real parts and unstable if at least one root has positive real part.Therefore, in order to study the local stability of the zero equilibrium of system 2.3 , we need to investigate the distribution of the roots of 2.3 .When τ 0, characteristic equation 2.3 yields Let y λ 1, 2.4 reduces to

2.6
From Cardano formula for the third-degree algebra equation, we have the following lemma.
(1) If Δ > 0, then 2.5 has a real root α β and a pair of conjugate complex roots 5 has a simple root 2α and a multiple root −α with the multiplicity of 2. Furthermore, the roots of 2.4 are given by λ 1 −1 2α and λ 2,3 −1 − α.Meanwhile, if w 1 w 2 0, that is, α 0, then 2.4 has a multiple root −1 with the multiplicity of 3.
Let z λ 1 e λτ , then 2.3 becomes We notice that 2.7 and 2.

2.17
We have 2.18 This completes the proof.
For convenience, we let where 2.20 and τ n j is defined in 2.13 .
Applying Lemmas 2.1-2.5 and Corollary 2.4 of Ruan and Wei 13 , we have the following results.

Lemma 2.6. All roots of 2.3 have negative real parts if one of the following holds:
Lemma 2.7.Suppose that one of the following hypothesis is satisfied: Then, there exists a sequence values of τ defined by 2.19 such that all roots of 2.3 have negative real parts for all τ ∈ 0, τ 0 , and 2.3 has at least one root with positive real part when τ > τ 0 , and 2.3 exactly has a pair of purely imaginary roots ±iω n n 1, 2, 3 when τ τ n j n 1, 2, 3; j 0, 1, 2, 3, . . ., where ω n and τ τ n j are defined by 2.12 and 2.13 , respectively.From Lemmas 2.5-2.7 and the Hopf bifurcation theorem for functional differential equations in 14 , we have the theorem.Theorem 2.8.(1) If one of the hypothesis H 1 , H 2 , H 3 is satisfied, then the zero solution of system 1.3 is asymptotically stable for all τ ≥ 0.

Direction of Hopf Bifurcations and Stability of the Bifurcating Periodic Orbits
In this section, we will study the direction of the Hopf bifurcation and stability of bifurcating periodic solutions by using the normal theory and the center manifold theorem due to Hassard et al. 12 . Let where Setting τ ν τ j , we know that ν 0 is Hopf bifurcation value of system 3.1 .
By the Riesz representation theorem, there exists a function η θ, ν of bounded variation for θ ∈ −1, 0 , such that In fact, we can choose where δ denotes the Dirac delta function.For φ ∈ C −1, 0 , R 3 , define where and a bilinear inner product where η θ η θ, 0 .Then, A 0 and A * are adjoint operators.By the discussion in Section 2, we know that ±iω 0 τ j are eigenvalues of A 0 and A * corresponding to iω 0 τ j and −iω 0 τ j , respectively.

3.13
In order to assure q * s , q θ 1, we need to determine the value of D. By 3.10 , we have 3.14 Therefore, we can choose D as D 1 1 q 1 q * 1 q 2 q * 2 τ j f 0 e iω 0 τ j w 1 q * 2 q 1 w 2 q 1 q * 2 q 2 q * 1 .
, satisfying |z n | > 1, then 2.11 has a positive root given by 5 have the same coefficients.Denote the three roots of 2.7 by z n R n iI n n 1, 2, 3 .Hence, 2.3 is equivalent to Lemma 2.4.If Δ ≥ 0, then 2.11 has at most two positive roots.If Δ < 0, then 2.11 has at most three positive roots.Without loss of generality, one assumes that 2.11 has three positive roots ω