Conditions for the global asymptotic stability of delayed artificial neural network model of n (≥3) neurons have been derived. For bifurcation analysis with respect to delay we have considered the model with three neurons and used suitable transformation on multiple time delays to reduce it to a system with single delay. Bifurcation analysis is discussed with respect to single delay. Numerical simulations are presented to verify the analytical results. Using numerical simulation, the role of delay and neuronal gain parameter in changing the dynamics of the neural network model has been discussed.
1. Introduction
In recent years, neural networks (especially Hopfield type, cellular, bidirectional, and recurrent neural networks) have been applied successfully in many areas such as signal processing, pattern recognition, and associative memories. In most of the research works, stability of the designed neural network is an important step of analyzing the dynamics. Chaos plays an important role in human brain cognitive functions related to memory process. For example, chaotic behavior has been observed in nerve membranes by electrophysiological experiments on squid giant axons [1–3] and in measurements of brain electroencephalograms (EEG) [4, 5]. At first, Aihara et al. [6] introduced chaotic neural network models in order to simulate the chaotic behavior of biological neurons. Both the network and its component neuron are responsible for chaotic dynamics if suitable parameter values are chosen [6, 7]. The investigation of chaotic neural networks is of practical importance and many interesting results have been obtained so far (see [8–11] and the references therein). The control of chaotic behavior in chaotic neural networks is an important problem to apply them in information processing [12]. The first chaos control was proposed by Ott et al. (the OGY method) [13]. Since this pioneer work of OGY, various methods such as the occasional proportional feedback (OPF) method [14], continuous feedback control [15], and pinning method [16] have been proposed for chaos control.
Many research works have been done to know the effect of time delays in neural system [17–19]. The malfunctioning of the neural system is often related to changes in the delay parameter causing unmanageable shifts in the phases of the neural signals. In the olfactory system, the phase transition has the appearance of a change in the EEG from a chaotic, aperiodic fluctuation to a more regular nearly periodic oscillation. In fact, neural network with delay can actually synchronize more easily controlling chaotic behavior of the system [10, 20]. Besides, change of neuronal gain parameter causes a change of all connectivity weights and therefore affects the dynamical behavior of the network [21].
For this reason, we are motivated to study effectiveness of time delay as well as neuronal gain parameters in changing the dynamics of an artificial neural network model. Using Lyapunov functional method, we studied the global stability analysis of the system and obtained sufficient criteria involving synaptic weights. We investigated the system numerically without time delay and with time delay. Chaotic behavior of the system without delay is controlled as the interconnection transmission delay is introduced in the model. Hopf-bifurcation analysis of the system with respect to time delay as parameter is discussed. Analytical results are verified using numerical simulations to show the reliability and effectiveness of the model. Numerical simulations of the model (using time series, phase portraits, and bifurcation diagrams) are presented showing changes of dynamics of the system. Chaotic behavior is observed when a gradual increase of slope of activation functions is made. Finally, some concluding remarks have been drawn on the implication of our results in the context of related work mentioned above.
2. Mathematical Model with Time Delay and Global Stability
We consider an artificial n-neural network model (n≥3) of time delayed connections between the neurons using system of delay differential equations:
(1)dy1dt=-α1y1(t)+f1[∑j=2nwj1yj(t-τj)]dyjdt=-αjyj(t)+fj[y1(t-τ1)]j=2,3,…,n(2)fk(Y)=11+e-βk(Y-θk),k=1,2,…,n,
where fk(Y) is the response functions and θk and βk are the threshold and the slope (neuronal gain parameter) of the response function of the neuron k, respectively. The value of this sigmoidal type of response function is always nonnegative, bounded by 0 and 1. wj1 denotes the weight of the synaptic connection from the first neuron to thejth neuron, αk is the decay or degradation rate parameter, and τk is the signal transmission delay and nonnegative constant.
In the following, we assume that each of the relations between the output of the cell f and the state of the cell possesses the following properties.
f is bounded function in R;
f is continuous and differentiable nonlinear function.
To clarify our main results, we present the following lemma and proof under more general conditions.
Lemma 1.
For the delayed nonlinear system (1), suppose that the output of the cell fsatisfies the hypotheses (H1) and (H2) above. Then all solutions of (1) remain bounded for [0,+∞).
Proof.
It is easy to verify that all solutions of (1) satisfy differential inequalities of the form
(3)-αkyk(t)-γk≤y˙k(t)≤-αkyk(t)+γk,
where γk=sup|fk[∑i=1nwikyi(t-τi)]|, k=1,2,…,n; and
(4)(wik)n×n=(0w21w31⋯wn1100⋯0100⋯0⋯⋯⋯⋯⋯1000)n×n.
Using (3), one can easily prove that solutions of (1) remain bounded on [0,+∞). This completes the proof.
By coordinate translation vk(t)=yk(t)-yk*, k=1,2,…,n, and assuming τ1=τ2=⋯=τn=τ, (1) can be written as
(5)dv1dt=-α1v1(t)+g1[∑j=2nwj1vj(t-τ)]dvjdt=-αjvj(t)+gj[v1(t-τ)]j=2,3,…,n,
where
(6)g1[∑j=2nwj1vj(t)]=f1[∑j=2nwj1(vj(t)+yj*)]-f1[∑j=2nwj1yj*]gj[v1(t)]=fj[v1(t)+y1*]-f1[y1*].
Clearly, (0,0,…,0)T is an equilibrium of (5). To prove the global asymptotic stability of y* of (1), it is sufficient to prove global asymptotic stability of the trivial solution of (5).
This section derives the criteria for the global asymptotic stability of the trivial equilibrium of the network by constructing a suitable Lyapunov functional.
Theorem 2.
If (i) 2α1>∑k=1nαk and (ii) 2∑j=2nαj2>[∑j=2nαj+α1∑j=2nwj12], then the trivial solution of (5) is globally asymptotically stable for any delay.
Proof.
We consider Lyapunov functional defined by
(7)V(t)=V(r)(t)=α1v12(t)+α1∫t-τtg12[∑j=2nwj1vj(r)]dr+∑j=2nαjvj2(t)+∑j=2nαj∫t-τtgj2[v1(r)]dr∴V˙(t)=2α1v1(t){-α1v1(t)+g1[∑j=2nwj1vj(t-τ)]}+α1g12[∑j=2nwj1vj(t)]-α1g12[∑j=2nwj1vj(t-τ)]+2∑j=2nαjvj(t){-αjvj(t)+gj[v1(t-τ)]}+∑j=2nαjgj2[v1(t)]-∑j=2nαjgj2[v1(t-τ)]≤-2α12v12(t)+α1v12(t)+α1g12[∑j=2nwj1vj(t)]-2∑j=2nαj2vj2(t)+∑j=2nαjvj2(t)+∑j=2nαjgj2[v1(t)].
As g1 and gj(j=2,3,…,n) satisfy the propositions on f1 and fj, we have the following expressions:
(8)g1[∑j=2nwj1vj(t)]=pj1(t)[∑j=2nwj1vj(t)],
where pj1(t)=∫01g1′(r∑j=2nwj1vj(t))dr,
(9)gj[v1(t)]=qj1(t)v1(t),
where qj1(t)=∫01gj′(rv1(t))dr, and there exist p*, q*∈(0,1] such that pj1(t)≤p*≤1, qj1(t)≤q*≤1, j=2,3,…,n.
Thus
(11)dVdt≤-2α12v12(t)+α1v12(t)+v12(t)∑j=2nαj-2∑j=2nαj2vj2(t)+∑j=2nαjvj2(t)+α1∑j=2nwj12vj2(t)=[-2α12+∑k=1nαk]v12(t)+[α1∑j=2nwj12+∑j=2nαj-2∑j=2nαj2]vj2(t)∴dVdt≤-[2α12-∑k=1nαk]v12(t)-∑j=2n[2αj2-αj-α1wj12]vj2(t).
So if 2α1>∑k=1nαk and 2∑j=2nαj2>[∑j=2nαj+α1∑j=2nwj12], then dV/dt<0 when v≠0; hence the trivial solution of (5) is globally asymptotically stable.
3. Bifurcation Analysis
As it is very difficult to study the dynamics of the proposed model analytically, we will study bifurcation analysis for n=3 only with three distinct delays. Hence our mathematical model will take the form as
(12)dy1dt=-α1y1(t)+f1[w21y2(t-τ2)+w31y3(t-τ3)]dy2dt=-α2y2(t)+f2[y1(t-τ1)]dy3dt=-α3y3(t)+f3[y1(t-τ1)].
For simplicity, the following set of new state variables are introduced:
(13)x1(t)=y1(t-(τ2+τ3)),x2(t)=y2(t-τ2),x3(t)=y3(t-τ3).
Also, let σ=τ1+τ2+τ3.
Now, system (12) takes the following equivalent form with single delay (σ) as
(14)x˙1=-α1x1(t)+f1[w21x2(t)+w31x3(t)]x˙2=-α2x2(t)+f2[x1(t-σ)]x˙3=-α3x3(t)+f3[x1(t-σ)].
Linearizing the system about the steady state E*=(x1*,x2*,x3*), we get
(15)U˙=(u˙1u˙2u˙3)=E(u1u2u3),
where u1=x1-x1*, u2=x2-x2*, and u3=x3-x3*.
The characteristic equation associated with the linearized system (14) is
(16)D(λ,τ)=λ3+A1λ2+A2λ+A3-(A4λ+A5)e-λσ=0,
where
(17)A1=α1+α2+α3,A2=α1α2+α2α3+α3α1,A3=α1α2α3,A4=P(w21M+w31N),A5=P(w21α3M+w31α2N),M=β2α2x2*(1-α2x2*),N=β3α3x3*(1-α3x3*),P=β1α1x1*(1-α1x1*),
and β1>0 so M>0, N>0, and P>0.
Let iω(ω>0) be a root of (16); then
(18)-iω3-A1ω2+iA2ω+A3-(iA4ω+A5)(cosωσ-isinωσ)=0.
Eliminating the harmonic terms in (18), we have
(19)D(ω)=ω6+(A12-2A2)ω4+(A22-2A1A3-A42)ω2+A32-A52=0.
The existence of unique σc is given by
(20)σc=1ωtan-1(A5ω(ω2-A2)-ωA4(A1ω2-A3)A5(A3-A1ω2)+ω2A4(A2-ω2))+ηπω,iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiη=0,1,2,….
So for different values of η, we will get different values of σc. For simplicity, we will denote the minimum value of σc by σ0 and let ω0 be the positive and simple root of (18). By differentiating λ with respect to σ at σ=σc in (16), we will get
(21)λ′(σc)=λ(A4λ+A5)(3λ2+2A1λ+A2)eλτ-A4+σc(A4λ+A5).
Next, based on the above results, a theorem is established.
Theorem 3.
E* is asymptotically stable for σ=0, and it is impossible that it remains stable for all σ>0. Hence, there exists a σc>0, such that, for σ<σc, E* is asymptotically stable and, for σ>σc, E* is unstable and as σ increases through σc, E* bifurcates into small amplitude periodic solutions of Hopf type [22].
4. Numerical Simulation
In this section, the network consisting of five neurons with connections as depicted in Figure 1 will be considered. For bifurcation results, we assume n=3. Similar results can be obtained also for n=6. We used Matlab 7.10.0.499 for simulation of numerical examples.
Structure of the five neural networks investigated.
4.1. Chaos Control
We consider the system n=5 without any time delay (see Example 1). This system (22) shows chaotic nature (see Figure 2(a)) which is controlled as transmission delay is introduced (see Example 2). Here, we have considered time delay τ as system parameter keeping other system parameter values as in system (22). Figure 2(b) shows periodic behavior in presence of delay (τ=5). Bifurcation diagram with respect to time delay τ is also illustrated in Figure 3.
Time series (a) of the system (22) without delay showing chaotic behavior; (b) control of chaos of the system (23) can be seen in solution trajectory with showing periodic behavior when delay τ=5.
Bifurcation diagram with respect to time delay τ showing control of chaos of system (22).
Here, we consider the model (5) when n=5 and the parameter values are so chosen (α2=4.5) which satisfy the global stability criteria independent of delay. Considering τ=15, we observe stable behavior of the system in Figure 4. But considering the same time delay (τ=15), the numerical simulation of Example 3 violating the conditions of global stability criteria shows the chaotic behavior in Figure 5.
Time series and corresponding phase portrait showing stable behavior satisfying global stability conditions when delay τ=15.
Time series and corresponding phase portrait showing chaotic behavior when delay τ=15 violating global stability conditions.
4.3. Bifurcation Results4.3.1. Bifurcation with Respect to Total Time Delay σ
We consider model (14) with total time delay σ(=τ1+τ2+τ3) in the following example.
Example 4.
Consider
(25)x˙1=-0.65x1(t)+1[1+exp(-7(x2(t)-0.5x3(t)-0.6))]x˙2=-0.45x2(t)+1[1+exp(-10(x1(t-σ)-0.3))]x˙3=-0.1x3(t)+1[1+exp(-13(x1(t-σ)-0.7))].
Substituting the system parameters into (19) and (20) for η=0, we obtain ω0=1.3, σ0=0.36, and λ′(σ0)=0.1-0.13i and the system has a positive equilibrium E*(0.64,2.15,3.14). When σ=0, the positive equilibrium E* is asymptotically stable and it remains stable when σ<σ0 as is illustrated by the computer simulations (see Figure 6(a)) and system (25) is unstable for all σ>σ0 (see Figure 6(b)). Bifurcation diagram with respect to time delay σ showing critical value is illustrated in Figure 7(a).
Solution trajectory showing (a) asymptotic stable behavior when σ=0.2<0.36=σ0; (b) periodic behavior with σ=0.8>0.36=σ0.
Bifurcation diagram (a) of the system (25) with respect to time delay σ showing critical value σ0=0.36; (b) of the system (26) with respect to β1 when τ=12.
4.3.2. Bifurcation with Respect to Neuronal Gain Parameter β1
We consider model (12) with the same time delay in the following example.
Example 5.
Consider
(26)y˙1=-0.65y1(t)+1[1+exp(-7(y2(t-τ)-0.5y3(t-τ)-0.6))]y˙2=-0.45y2(t)+1[1+exp(-10(y1(t-τ)-0.3))]y˙3=-0.1y3(t)+1[1+exp(-13(y1(t-τ)-0.7))].
Bifurcation diagram with respect to β1 is demonstrated in Figure 7(b). Gradual increase of slope β1 of the activation function causes periodic oscillation and then chaos through period doubling of the system (26).
5. Conclusion
In this paper, we studied the global stability of artificial neural network model of n-neurons with time delay and obtained the criteria of involving the synaptic weight and decay parameters, independent of delay. The system shows chaotic behavior without delay and introduction of delay plays a vital role in controlling chaos in the system. Also bifurcation analysis of the model with respect to time delay shows transition from stable to unstable (i.e., resting to rhythmic) behavior increasing the value of delay. We have also illustrated the effect of changing slope of the sigmoidal activation function (gain parameter β1) in the dynamics of the neural network. Neural gain can be thought of as an amplifier of neural communication. When gain is increased, excited neurons become more active and inhibited neurons become less active [21]. The numerical simulation shows that the system dynamics undergo changes from stable to periodic and then to chaotic behavior through period doubling as the gain parameter β1 is increased. A similar kind of behavior can be observed also for other neuronal gain parameters. The network with very small gain and small delay may not be very useful for implementation in models of oscillation generators and associative memories.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
AiharaK.MatsumotoG.HoldenA. V.Chaotic oscillations and bifurcations in squid giant axons1986257269MR848814MatsumotoG.AiharaK.HanyuY.TakahashiN.YoshizawaS.NagumoJ.-I.Chaos and phase locking in normal squid axons1987123416216610.1016/0375-9601(87)90696-72-s2.0-0001690795KaplanD. T.ClayJ. R.ManningT.GlassL.GuevaraM. R.ShrierA.Subthreshold dynamics in periodically stimulated squid giant axons199676214074407710.1103/PhysRevLett.76.40742-s2.0-0000058225BabloyantzA.DestexheA.Low-dimensional chaos in an instance of epilepsy198683103513351710.1073/pnas.83.10.35132-s2.0-0022530614FreemanW. J.Simulation of chaotic EEG patterns with a dynamic model of the olfactory system1987562-313915010.1007/BF003179882-s2.0-0023186918AiharaK.TakabeT.ToyodaM.Chaotic neural networks19901446-733334010.1016/0375-9601(90)90136-CMR10451282-s2.0-0001553201AdachiM.AiharaK.Associative dynamics in a chaotic neural network1997101839810.1016/S0893-6080(96)00061-52-s2.0-0031033677DasA.RoyA. B.DasP.Chaos in a three-dimensional general model of neural network200212102271228110.1142/S0218127402005820MR19412822-s2.0-0036816233LiX.BohnerM.Exponential synchronization of chaotic neural networks with mixed delays and impulsive effects via output coupling with delay feedback2010525-664365310.1016/j.mcm.2010.04.011MR26617512-s2.0-77954176141DasP.KunduA.Bifurcation and chaos in delayed cellular neural network model2014221922410.4236/jamp.2014.25027KunduA.DasP.RoyA. B.Complex dynamics of a four neuron network model having a pair of short-cut connections with multiple delays201372364366210.1007/s11071-012-0742-2MR30469192-s2.0-84880699501HeG.CaoZ.ZhuP.OguraH.Controlling chaos in a chaotic neural network20031681195120010.1016/S0893-6080(03)00055-82-s2.0-0041833487OttE.GrebogiC.YorkeJ. A.Controlling chaos199064111196119910.1103/PhysRevLett.64.1196MR1041523ZBL0964.375012-s2.0-4243489552HuntE. R.Stabilizing high-period orbits in a chaotic system: the diode resonator199167151953195510.1103/PhysRevLett.67.19532-s2.0-12044250986GangH.ZhilinQ.Controlling spatiotemporal chaos in coupled map lattice systems1994721687110.1103/PhysRevLett.72.682-s2.0-12044257605ChenL.AiharaK.Global searching ability of chaotic neural networks199946897499310.1109/81.780378MR17027282-s2.0-0032673995HilbornR. C.2000Oxford, UKOxford University PressMR1263025XiaoM.ZhengW. X.CaoJ.Bifurcation and control in a neural network with small and large delays20134413214210.1016/j.neunet.2013.03.0162-s2.0-84876452634SongY.HanM.WeiJ.Stability and Hopf bifurcation analysis on a simplified BAM neural network with delays20052003-418520410.1016/j.physd.2004.10.010MR21130252-s2.0-12244261643AtayF. M.JostJ.WendeA.Delays, connection topology, and synchronization of coupled chaotic maps2004921410.1103/PhysRevLett.92.1441012-s2.0-2542431567EldarE.CohenJ. D.NivY.The effects of neural gain on attention and learning20131681146115310.1038/nn.34282-s2.0-84880911656HassardB. D.KazarinoffN. D.WanY. H.1981Cambridge, UKCambridge University PressMR603442