Nonlinear Dynamics and Chaos in Fractional-Order Hopfield Neural Networks with Delay

A fractional-order two-neuron Hopfield neural network with delay is proposed based on the classic well-known Hopfield neural networks, and further, the complex dynamical behaviors of such a network are investigated. A great variety of interesting dynamical phenomena, including single-periodic,multiple-periodic, and chaoticmotions, are found to exist.The existence of chaotic attractors is verified by the bifurcation diagram and phase portraits as well.


Introduction
Fractional calculus, which mainly deals with derivatives and integrals of arbitrary order, was firstly introduced 300 years ago.However, it is only in recent decades that fractional calculus is applied to physics and engineering [1][2][3].The main advantage of fractional-order models in comparison with its integer-order counterparts is that fractional derivatives provide an excellent tool in the description of memory and hereditary properties of various processes.In fractional calculus, a generalized capacitor, called "fractance, " is often considered to be the main operator.It is actually an electrical circuit in which its voltage and current are related by the fractional-order differential equation [4].
Chaos theory has been extensively investigated in various fields of research after the first observation of chaotic attractors in Lorenz system.Recently, study on the complex dynamical behaviors of fractional-order systems has become a hot research topic due to the fact that fractional-order systems show higher nonlinearity and more degrees of freedom in the models, and therefore fractional-order chaotic systems are considered to have the potential ability of improving the security of chaotic communication systems [5].It has been known that chaos in many well-known integer-order chaotic systems will remain when the orders become fractional, and a great number of fractional-order chaotic systems have been proposed as a consequence [6][7][8][9][10][11][12][13][14].Moreover, chaotic behaviors have also been found to exist in some discontinuous systems with fractional derivatives [15].However, it is worthwhile to note that none of the aforementioned fractional-order chaotic systems are time-delayed systems.
On the other hand, the dynamics of delayed neural networks (DNNs) with traditional integer-order derivatives have been extensively studied both in theory and applications.[16][17][18][19][20][21][22][23][24][25].It has been reported that DNNs can really display quite rich dynamical behaviors.For instance, Lu studied the complex dynamics of a DNN of Hopfield-type with two neurons and concluded that for a certain set of system parameters such a network does exhibit chaotic attractors [23].The complex dynamics of a delay-free neural network of Hopfield-type with four neurons have been investigated in [24], and hyperchaotic attractors are found to exist for some particular set of weight matrices.Chaotic attractors are also known to exist in delayed cellular neural networks [25].
However, the dynamical analysis of fractional-order neural networks is a very recent and promising research topic.
Actually, if we replace the common capacitor in the integerorder Hopfield neural networks (HNNs) by the aforementioned "fractance, " we obtain the model of fractional-order HNNs.The incorporation of fractional derivatives into the network is a great improvement in modeling.It has been pointed out that fractional differentiation provides neurons with a fundamental and general computation that can contribute to efficient information processing [26].More recently, efforts have been made to investigate the complex dynamics of fractional-order neural networks.In [27], Arena et al. firstly introduced a cellular neural network (CNN) with fractional-order cells.In [28], Petráš presented a fractional three-cell CNN which exhibits limit cycle and stable orbit for different parameter values.A fractional two-cell chaotic CNN was reported in [29] and the corresponding strange attractor was shown.In [30], a fractional-order four-cell CNN has been presented and hyperchaotic attractors have been displayed.In [31], the dynamics of fractional-order delay-free HNNs, including stability and multistability, bifurcations, and chaos, have been investigated.Time-delay is ubiquitous in physical systems, control systems, and biological systems due to finite switching speeds of amplifiers, finite signal propagation time in biological networks, and so on.Chaotic attractors in a fractionalorder CNN with time-delay have been observed when the fractional-order  ≥ 0.1 [32].The analysis of delay dynamics in networked control systems has been made in [33].Unfortunately, little effort has been devoted to the complex dynamics of delayed fractional-order neural networks of Hopfield-type so far.This is a new subject and needs to be explored in depth.The main aim of this paper is to investigate the complex dynamics of fractional-order HNNs with time delay; a great variety of interesting dynamical behaviors, such as periodic and chaotic will be observed and the effect of the time delay will also be discovered.The existence of chaotic attractors is verified by bifurcation diagram and phase portraits, respectively.
The rest of this paper is organized as follows.In Section 2, definition for fractional derivatives and a modified numerical algorithm for solving delayed fractional-order differential equations are presented.In Section 3, a delayed fractionalorder HNN with two neurons is proposed.Bifurcation analysis is made in Section 4. The lowest order, as well as the highest order, for chaos to exist is determined and the effect of the time delay is revealed.Finally, some concluding remarks are reported in Section 5.

Preliminaries and Notations
There are three most frequently used definitions for fractional derivatives, that is, Riemann-Liouville, Grünwald-Letnikov, and Caputo's definitions.The main advantage of Caputo definition is that the initial conditions for fractional-order differential equations with Caputo derivatives take on the same form as for integer-order differential equations [1].And therefore, this paper is based on Caputo's definition.
Definition 1 (see [1]).The Caputo fractional derivative of order  of a continuous function  :  + →  is defined as follows: where Γ is Γ-function, and The numerical calculation of fractional-order differential equations (FDEs) is not as simple as that of ordinary differential equations (ODEs).Due to the nonlocal character of fractional derivative operator, the numerical methods for solving ODEs have to be modified to cater for solving FDEs.In [34,35], a method to numerically solve FDEs has been proposed based on the Adams-Bashforth-Moulton predictorcorrector scheme.Furthermore, an algorithm for solving fractional-order delayed differential equations (FDDEs) in the form of    () = (, ( − )) is presented in [36].We modify the predictor-corrector algorithm proposed in [36] such that it is appropriate to solve FDDEs in the form of (3) as below.
Consider the FDDEs described by It should be noted that  is assumed to satisfy the Lipschitz condition with respect to its second and third variables, respectively, and  is assumed to be continuous in (3) in order to ensure the existence of solutions.For more details, the reader can consult [37] and the references cited therein.
Applying fractional integration    on both sides of (3), we obtain Discretizing (4) for uniform grid,   = ℎ ( = −, − + 1, . . ., −1, 0, 1, . . ., ), in which  and  are integers and satisfy  = ℎ and  = ℎ.Let and denote Suppose that we have obtained the approximations for  ℎ (  ) ( = 1, 2, . . ., ), and now we wish to calculate  ℎ ( +1 ) by using We use approximations  ℎ (  ) for (  ) in (7).Thus, the modified predictor-corrector algorithm can be presented as follows: where and   ℎ ( +1 ) is the predicted value of  ℎ ( +1 ) which can be determined by where Remark 2. It should be noticed that the algorithm proposed in [36] can be used to calculate the FDDEs in the form of    () = (, ( − )), where the variable () does not explicitly appear in the right-hand side function .However, when  is also a function of (), that is, the FDDEs described by    () = (, (), ( − )), the algorithm proposed in [36] fails to be used in this case.For this reason, we modify the algorithm such that it is appropriate to solve FDDEs in the form of (3).Note that (10) is used indispensably to predict the value of  ℎ ( +1 ) as   ℎ ( +1 ).This is the main difference between the modified algorithm and the algorithm proposed in [36].

Description of the Delayed Fractional-Order HNNs
In this paper, we consider the fractional-order HNNs with time delay described by where  denotes the Caputo fractional derivative of order ,   > 0,   and   are real numbers,  ≥ 0 represents the time delay, and   are external inputs.The activation function () is chosen as () = tanh (), and   () =   (),  ∈ [−, 0], is the initial condition.
It should be noted that when  = 1, (12) is reduced to the classical integer-order delayed HNNs.The complex dynamical behaviors of such a network with two neurons have been investigated in [23] and single-periodic, multiple-periodic, and chaotic motions have been revealed.In this paper, we study the fractional-order case, that is, the complex dynamics of system (12) with two neurons.
When the fractional order  = 0.82, Figure 1(a) shows the two separate period-one orbits with two groups of initial conditions  1 () = 0.4,  2 () = 0.6 and  1 () = −0.4, 2 () = −0.6 for  ∈ [−1, 0], respectively.It can be easily found from Figure 1(a) that one period orbit oscillates around the equilibrium point  * 1 = (−0.37614,2.35978); the other oscillates around the equilibrium point  * 3 = (0.37614, −2.35978).With the increase of the derivative order, period-two orbits and period-four orbits appear.Figure 1(b) shows two separate period-two orbits with order  = 0.84 and Figure 1(c) shows two separate period-four orbits with order  = 0.85.If we continue to increase the derivative order, then system (13) begins to exhibit chaotic behavior.Figure 1(d) shows two separate coexisting single-scroll chaotic attractors with order  = 0.90.It should be noted that Figures 1(b), 1(c), and 1(d) take on the same initial values as those in Figure 1(a).
If we further increase the derivative order to 0.92, two separate single-scroll chaotic attractors merge into a doublescroll chaotic attractor.Figure 2(a) shows a fully developed double-scroll chaotic attractor with order  = 0.92 and the initial condition is set as  1 () = 0.4,  2 () = 0.6 for  ∈ [−1, 0].When the derivative order is increased to  = 0.99, Figure 2(b) shows a periodic orbit with the initial condition  1 () = 0.4,  2 () = 0.6 for  ∈ [−1, 0].That is to say, for fractional-order nonlinear system with time delay, there is a chance to be periodic after it has already entered the chaotic domain.
We summarize the considered cases in Figures 1 and 2 in Table 1.We can find from Table 1 that when keeping the values of  22 and  22 fixed, system (13) firstly undergoes the process of period-doubling bifurcation with the increase of  to some extent and then tends from disorder to order when the value of  continues to increase.This indicates that the dynamical characteristics of fractional-order nonlinear systems with delays are completely different from fractionalorder nonlinear systems without delays.Remark 3. It is generally known that chaotic behavior in fractional-order nonlinear systems without time delays is preserved when the derivative order is greater than a certain value (called as the lowest order).However, an interesting phenomenon for delayed fractional-order nonlinear system is observed in the simulation; different from fractional-order system without time delay, chaotic phenomena exist only for a certain interval of derivative order, that is to say, a lowest order as well as a highest order has been found to exist in order to ensure the occurrence of chaos.As long as the derivative order is beyond these two bounds, the chaotic attractor will vanish.
In order to confirm the period-doubling bifurcation process, we draw the bifurcation diagrams of  1 with respect to the derivative order . Figure 3 shows a bifurcation process when  ranges from 0.8 to 1 with  22 = 2.0,  22 = −1.5.It can be determined from Figure 3 that a periodic window is found when  is about 0.99.It confirms what we have observed in Figures 1 and 2.
In what follows, we modify only the values of  22 and  22 into  22 = 3.0,  22 = −2.5 to investigate the dynamics of system (13).Obviously, the equilibrium points of system (13) do not change.It can be easily seen that with the increase of the system order, system (13) undergoes a similar process of period-doubling bifurcation.
It should also be noted that another interesting phenomenon is observed; that is, even when the system enters the chaotic domain and if we continue to increase the fractional Advances in Mathematical Physics order, now the system will exhibit different patterns of chaotic attractors.Figure 5 shows different fully developed doublescroll chaotic attractors with parameters  22 = 3.0,  22 = −2.5 when system order is further increased.Similarly, we summarize the considered cases in Figures 4 and 5 in Table 2.We can see from Table 2 that when keeping the values of  22 = 3.0 and  22 = −2.5 fixed, system (13) undergoes the process of period-doubling bifurcation with the increase of  and then the chaotic behavior is preserved although the patterns of the chaotic attractors are different.Similarly, in order to confirm the period-doubling bifurcation process, we draw the bifurcation diagrams of  1 with respect to derivative order .

Conclusion
In this paper, the complex dynamical behaviors of delayed fractional-order Hopfield-type neural networks with two neurons are detailedly investigated with the aid of numerical simulations.Different varieties of interesting dynamical behaviors, including single-periodic, multiple-periodic, and double-scroll chaotic attractors, have been discovered.It should be pointed out that different from fractional-order nonlinear systems without time delay, a lowest order as well as a highest order has been found to exist in order to ensure the emergence of chaos.The existence of chaotic attractors is verified with bifurcation diagram and phase portraits.The novel fractional-order chaos provides a new chaos generator and can be applied in many engineering applications, especially in the secure communication.
Figure 6  demonstrates bifurcation process when  ranges from 0.75 to 1 with  22 = 3.0,  22 = −2.5, which confirms what we have observed in Figures4 and 5 .