A novel chaotic neural network using memristive synapse with applications in associative memory

. Chaotic Neural Network, also denoted by the acronym CNN, has rich dynamical behaviors that can be harnessed in promising engineering applications. However, due to its complex synapse learning rules and network structure, it is di ﬃ cult to update its synaptic weights quickly and implement its large scale physical circuit. This paper addresses an implementation scheme of a novel CNN with memristive neural synapses that may provide a feasible solution for further development of CNN. Memristor, widely known as the fourth fundamental circuit element, was theoretically predicted by Chua in 1971 and has been developed in 2008 by the researchers in Hewlett-Packard Laboratory. Memristor based hybrid nanoscale CMOS technology is expected to revolutionize the digital and neuromorphic computation. The proposed memristive CNN has four signiﬁcant features: (cid:3) 1 (cid:4) nanoscale memristors can simplify the synaptic circuit greatly and enable the synaptic weights update easily; (cid:3) 2 (cid:4) it can separate stored patterns from superimposed input; (cid:3) 3 (cid:4) it can deal with one-to-many associative memory; (cid:3) 4 (cid:4) it can deal with many-to-many associative memory. Simulation results are provided to illustrate the e ﬀ ectiveness of the proposed scheme.


Introduction
Over the last three decades, chaos has been extensively studied by many researchers to understand complex dynamical phenomenon of nonlinear circuits.It has been found that chaotic dynamics may exist in biological neurons in human brain and play important role in associative memory 1, 2 .To mimic biological neuron, a chaotic neuron model has been proposed by Aihara et al. and subsequently many associative chaotic neural network models have been developed by other researchers 3-13 .In the traditional study of associative chaotic neural networks, researchers generally considered that each historic event works on the network with equal degree, which is,

CNN Based on Aihara's Model
The dynamical equation of a chaotic neuron model with continuous functions described by 1 , where x t 1 is the output of the chaotic neuron at the discrete time t 1, which is an analogue value between 0 and 1; f is a continuous output function, such as logistic function f y 1/ 1 e −y/ε with the steepness parameter ε; g is a function describing the relationship between the analog output and the refractoriness which can be defined as g x x; A t is the external stimulation; α, k, and θ are positive refractory scaling parameter, the damping factor, and the threshold, respectively.If defining the internal state y t 1 as we can reduce 2.1 to the following y t 1 ky t − αg f y t a t , x t 1 f y t 1 ,

2.3
where a t is called bifurcation parameter and defined by The neuron model with chaotic dynamics described above can be generalized as an element of neural network called chaotic neural network CNN .The dynamics of the ith chaotic neuron with spatiotemporal summation of feedback inputs and externally applied inputs in a CNN composed of M chaotic neurons and N externally applied inputs can be modeled as where the external inputs ξ i t 1 , the feedback inputs η i t 1 , and the refractoriness ζ i t 1 are defined as 2.6 -2.8 , respectively.
k d e A j t − d , 2.6 where V ij serves as the synaptic weight between the external inputs and the neurons.
Similarly, W ij indicates the synaptic weights between two neurons and is trained by Hebbian learning algorithm, such as, 2.9

Dynamical Behaviors Analysis
The firing rate of a neuron is a fundamental characteristic of the message that conveys to other neurons.It is variable and denotes the intensity of its activation state.As such, it ranges from near zero to some certain maximum depending on its need to convey a particular level of activation.Traditionally, it has been thought that most of the relevant information was contained in the average firing rate of the neuron.The firing rate is usually defined by a temporal average meaning the spikes that occur in a given time window.Division by the length of the time window gives the average firing rate 30 .Here, the average firing rate or excitation number of a single chaotic neuron depicted in 2.3 is defined as where h is a transfer function which represents waveform-shaping dynamics of the axon with a strict threshold for propagation of action potentials initiated at the trigger zone and assumed to be h x 1 for x ≥ 0.5 and h x 0 for x < 0.5.By adjusting the bifurcation parameter a t from 0 to 1, when the other parameters are set as k 0.7, α 1.0, ε 0.02, y 0 0.5, and the average firing rate is shown in Figure 1.It is a characteristic of chaotic systems that initially nearby trajectories separate exponentially with time.The Lyapunov exponent or Lyapunov characteristic exponent of a dynamical system is a quantity that characterizes the rate of separation of infinitesimally close trajectories.The rate of separation can be different for different orientations of initial separation vector.Thus, there is a spectrum of Lyapunov exponents-equal in number to the dimensionality of the phase space.It is common to refer to the largest one as the Maximal Lyapunov exponent MLE , because it determines a notion of predictability for a dynamical  system.Specifically, a positive MLE is usually taken as an indication that the system is chaotic 31 .
For quantitative analysis of the system 2.3 , the Lyapunov exponent λ is defined as follows, Response characteristics of this system are shown in Figure 2. From Figure 2 a , one can see the bifurcation structure clearly.Figure 2 b shows the Lyapunov exponent spectrum of this system.The values of the parameters employed are set as the same as those in Figure 1.

The Chaotic Neural Network with Memristive Neuronal Synapses
In this section, the memristor is introduced at first to describe its typical characteristics.Then a new chaotic neural network MCNN with memristive synapses is presented in detail.Finally, the associative memory abilities of the MCNN model are analyzed.

Memristor Model Description
All two-terminal nonvolatile memory devices based on resistance switching are memristors, regardless of the device material and physical operating mechanisms 15 .Similar to the resistor which relates voltage v and current i , capacitor which relates charge q and voltage v and inductor which relates flux ϕ and current i , the memristor relates the flux ϕ, and the charge q of a device.
The behaviors of a general memristive device can be described by a pair of coupled equations as following: where 3.1a is the i-v equation for the memristive device and w is an or a group of internal state variable which is determined by the excitation history of the device.A mathematical model of the typical HP TiO 2 -based memristor can be described by 16 where, M t M on w t /D M off 1 − w t /D is the total memristance, D and w indicate the thickness of the whole TiO 2 film and the high conductive region, respectively.If w 0 or w 1, memristor reaches the memristance limit value, M off or M on .w 0 denotes the initial value of w. μ v is average ion mobility parameter in units of m 2 s −1 V −1 .
Characteristics of the memristor have been analyzed through simulations.By applying a sinusoidal voltage, we can observe the symbolic pinched hysteresis loop in Figure 3 a confined to the first and the third quadrants of the i-v plane.The switching characteristic can be seen from this figure 28 , that is, memristor can switch between the high-resistance state "OFF" and the low-resistance state "ON" , which has been proposed in binary memory, Boolean calculation, crossbar array, and signal processing abundantly.It should be noted that the pinched hysteresis loop for i-v characteristic implies the device is a memristor, but this curve itself is useless as a model since its contour shape changes with both the amplitude and the frequency of any periodic sine wave like applied voltage or current source and thus cannot be used to predict the current response to arbitrarily applied voltage signals, and vice versa 15 .Moreover, it also can be found that under low-frequency AC excitation, there is an obvious i-v pinched hysteresis loop, but under high-frequency case, the hysteresis loop collapses and memristor degenerates into a fixed resistor.
One of the two ways to predict the response of the device is the q-ϕ constitutive relation in Figure 3 b .As seen in this figure, the memductance G ϕ t k at the time t t k is equal to the slope of the q-ϕ curve at ϕ ϕ t k , which is called small-signal memductance.Similarly, if the stimulation is a current source, the memristance M q t k at the time t t k is equal to the slope of the ϕ-q curve at q q t k , and called small-signal memristance.Assuming the constitutive relation curve is continuous at the time t t k and the stimulation has sufficiently small amplitude with a fixed frequency, which is called as small-signal condition 15 , the small-signal memductance or memristance approximately keeps constant being equal to the curve slope at the time t k over time and would be indistinguishable from a linear resistance.
Memristance versus state map is an equivalent representation as q-ϕ curve and also obeys Ohm's Law except that the memristance is not a constant, as illustrated by the example in Figure 3 c .The memristance decreases continuously with the flux increasing, and vice versa.When the flux reaches the limits, minimum or maximum, for a given memristive device, the memristance remain the ultimate value, M off or M on .
Figure 3 d clearly presents the relationship between the memristance and the applied voltage.By applying a series of small voltage pulses with the same magnitude but opposite polarity, we have observed the stair-like decrement or increment of memristance correspondingly.If the bias is a big stimulation, memristance changes continuously.Thus, one can tune and choose different memristances flexibly according to the requirements in different applications by controlling the amplitude, polarity, and duration of the stimulation.

Memristive Chaotic Neural Network
Based on previous works 10-13, 28 , we propose a new chaotic neural network with memristive neuronal synapses MCNN .This MCNN model has four distinguishing features from the traditional CNN: 1 continuously external input replaces initial input, 2 spatiotemporal effect is replaced with a new exponential decreasing effect, and 3 a part of summation replaces the whole spatiotemporal summation.In traditional associative CNN, researchers considered that each historic effect worked on the network with the same equal level, so they usually assumed decay parameters as constant values which are far from the truth in facts.4 The memristor is exploited for the synaptic weights.Memristors exhibit dynamical resistance modeled by the stimulation history, which is very similar to the biological neuron synapse and has been proposed in artificial neural network ANN as synaptic connection.The MCNN with the features above is more suitable to mimic the biological neuron.Moreover, since the memristor is a nanoscale device and it can simplify the synaptic circuit obviously, the MCNN is expected to be realized in physical implementation as a part of a VSLI chip.Figure 4 shows the structure of the proposed MCNN model that consists of three layers and all of the neurons are fully connected, which is similar to the Hopfield neural networks.
In the MCNN, the dynamics of the ith chaotic neuron in the αth layer is given as follows: 1 while t > t 0 , Here, x i α t 1 is the output of the chaotic neuron.L, N, and M are the number of layers, the chaotic neurons in the βth layer and in the αth layer; I i t − d is the input of the ith neuron in the αth layer; V i α t is the connection weight between the input and the neurons in the αth layer; W αβ ij t indicates the weight between the ith chaotic neuron in the αth layer and the jth neuron in the βth layer.The decay parameter and transfer function are defined as k u e −u and φ y 2/ 1 e −y/ε , respectively.For simplicity, g x x and h x x.Moreover, we utilize Hebbian learning regulation as learning algorithm of the synaptic weights.

Memristive Neuronal Synapse in MCNN
We present a simplified memristive synapse circuit using memristors as part of the neuronal synapse taking advantage of the continuously variable memristance and memory ability of memristors.Figure 5 shows the schematic diagram of the memristive synaptic circuit, in which the input voltage source V in is assumed as the output of a neuron, and the output of the synaptic circuit V out is connected to next neuron as the input signal after weighting.The memristor placed in the feedback path can be programmed by the constant programming current using the circuit's voltage to current conversion for a big programming voltage V pro .Assume that the Op-amp is ideal, and then the memristance after programming by a lowfrequency voltage V pro for a time width t p is given by

3.4
The synaptic weight W ij in 3.3 is realized by the gain of the amplifier: Different weight values can be obtained easily by changing memristance and the memristance can be set by applying different programming voltages.The synaptic weight keeps constant after the programming operation the programming voltage cut off based on the memory ability of memristor.If the synaptic weight should be a positive value, connecting a reversing amplifier to the memristive synaptic circuit may be an effective method.
Next, when the MCNN has been trained by setting its connection weights, it will work normally in some applications such as associative memory.As described in the front, memristors can be considered as a content resistor under a high frequency or very short operation time or a small enough external excitation.In this paper, each step of the programs in the simulations uses very short time, about 0.01 s in MATLAB, thus in order to simplify the experiments, we assume that the memristance approximately levels off at the value set in programming stage.At the practical level, for a real memristor, it has some certain threshold voltages presented in literatures 19 .Therefore, in physical implementation of the MCNN, this behavior can be realized by using some voltage-controlling solutions that enable memristance changeable depending on the operation requirement.

Associative Memory for MCNN
Here, we will primarily analyze the ability of associative memory of the memristive chaotic neural network by undertaking three important applications of the MCNN for examples.

Separation of Superimposed Patterns
Let us assume that a training pattern set {X, Y } is memorized in the neural network.When X is given to the network as an externally applied input continuously, the network searches around the input pattern, so pattern X is recalled.When the superimposed pattern X Y is given, the network searches around the X and Y .Due to the property that the chaotic neurons change their states by chaos, the X and the Y can be recalled, respectively.We can also simplify the separation procedure as follows:

One-to-Many Associations
Based on the property of separation of superimposed patterns, assume the following training set X 1 , Y 1 , Z 1 has been memorized in a three-layered MCNN, When X 1 is given to the first layer of the network continuously as input, the first layer searches and recalls the X 1 without doubt, the second and the third layer can search around Y 1 and Z 1 by chaotic itinerancy, respectively.So, X 1 , Y 1 , and Z 1 can be recalled in three layers, respectively, and one-to-many associations is realized.

Many-to-Many Associations
Let us consider the following training set When X 1 is given to the first layer of the network continuously, the internal state of the chaotic neuron in the second layer is determined by the following formula: where W αβ is the connection weight matrix between the first layer and the second layer, X 1 X T 3 Y 3 is the noise term, and Y 1 Y 2 is the superimposed term resulting from the common term X 1 .Similarly, the superimposed term Z 1 Z 2 appears in the third layer.The network searches around Y 1 and Y 2 by chaotic itinerancy in the second layer and searches around Z 1 and Z 2 in the third layer.Then, pattern X 1 , Y 1 , Z 1 and X 1 , Y 2 , Z 2 can be recalled at different time, realizing the many-to-many associations.

Simulation Results
In order to demonstrate the effectiveness and associative memory ability of the proposed MCNN, we perform a series of computer simulations.Since bipolar data −1 or 1 are used in patterns, the weight values can be calculated by simple derivation based on the Hebbian algorithm formula the first formula in 2.9 according to the number of the patterns to be remembered, that is, W ∈ {−N, −N 2, . . ., 0, . . ., N − 2, N}, where W is the weight value and N N > 1 is the number of the patterns.For example, if there are three patterns used in the weight computation at one time, the weight values belong to {−3, −1, 0, 1, 3}.
In the following experiments, the number of the patterns in the training set used for computing the weights is one, two, and three, respectively.So all of the weight values belong to {−3, −2, −1, 0, 1, 2, 3}, where, the value "0" means there is no connection between the two neurons, so we do not need to consider it.Moreover, it is known that the memristance must be positive, so the output of the synaptic circuit is negative seen from 3.5 .Since the weight values X and −X imply the same memristance state but with an opposite weight sign, so we use a reversing amplifier connected to the synaptic circuit to realize the positive weight.Therefore, for a given resistor R 10 kΩ, the memristor should be programmed to three states 30 kΩ, 20 kΩ, 10 kΩ, to realize the weights −3, −2, −1 and 3, 2, 1 .Figure 6 shows such three memristance states which we have got and the corresponding voltage pulses 1 V, 0.36 s , 1 V, 1.02 s , 1 V, 1.42 s .Next, by using the synaptic weights obtained from the memristive synaptic circuit, we will study the associative memory ability of the MCNN.

Separation of Superimposed Patterns of a MCNN
We have designed a MCNN containing 49 neurons to realize the separation of superimposed pattern and the training set is shown in Figure 7.The superimposed pattern A B is input to the network continuously and the number of output is 20.Network Parameters are set as a 9.8, kr1 0.9, kr2 0, kr3 0.1, kr4 0.9, V 200, θ 0, and logistic function with ε 0.02.This MCNN has a layer and the total valid weight values belong to {−3, −1, 1, 3}.
Figure 8 shows the output of the MCNN for separation of superimposed pattern, in which pattern A is separated from the superimposed pattern at step 6, 9, 13, and 16, and pattern B is separated at step 5, 8, 11, 14, and 17 successfully.This processing progress takes about 0.01 s in software MATLAB.
Figure 9 shows the training set A, a, 1 .After the whole pattern set is stored in the MCNN by Hebbian learning algorithm, when a pattern A is given to the network as an external input continuously, the whole pattern set can be recalled perfectly Figure 10 and realize one-to-many associations.This processing progress takes about 0.3 s in software MATLAB.

Many-to-Many Associations of a MCNN
A 3-layered 81-81-81 MCNN has been designed for many-to-many associations with a part of spatiotemporal summation t 0 3 the same as that in the experiments above .The parameters are a 10, kr1 0.95, kr2 0.1, kr3 0.1, kr4 0.95, V 100, θ 0,     are remembered in the first layer, the terms a, b, and c are remembered in the second layer, and 1, 2, and 3 in the third layer.In order to show the outputs clearly, we put the outputs from three layers at the same time together.Figure 12 shows the responses when A is given to the first layer as an external input, in which A, a, 1 appears 9 times and A, b, 2 appears 10 times in 30 steps of output.That implies the realization of the many-to-many associative memory of MCNN.This simulation takes about 0.4 s in MATLAB software.
It can be seen from the above experiments that the MCNN can realize associative memory successfully, which indicates it can perform the typical applications as the traditional CNN.Moreover, this MCNN has special characteristics over the traditional ones.First of all, it needs set its connection weights to realize memory ability for a CNN, which is usually called training operation.In order to perform different tasks, these weights should be updated correspondingly.Traditionally, changing these weights is difficult because the synaptic circuit is fixed and complex.In the meantime, this is also a key factor why CNN has not been practically implemented widely.Fortunately, the memristor possesses fast switching <10 ns , low energy ∼1 pJ/operation , high write endurance 10 10 , multiplestate operation, scalability, synapse-like behaviors, and CMOS compatibility.The size of a memristor is 1/10 of that of a transistor and requires much less energy, so the memristive synaptic circuit would be more simple and smaller than the traditional circuits.What is more important is it can be very easy to update the connection weights just by reprogramming the memristors in the synaptic circuits with proper programming voltages, which saves the big replacement of the synaptic circuit as in traditional CNN and makes the MCNN be expected to be realized in large scale physical circuit.

Conclusions
An implementation scheme for a new Chaotic Neural Network with memristive neuronal synapses MCNN is addressed in this paper.Exponential decay spatiotemporal effect and a part of spatiotemporal summation are considered in the MCNN, which is closer to biological neuron behavior and has higher efficiency in information processing such as associative memory.The characteristics of memristors are analyzed including the i-v characteristics, q-ϕ constitutive relation, and memristance versus state map with the relationship between the applied voltage and memristance.A memristive synaptic circuit is designed based on a variable gain amplifier.The weight value can be updated easily taking advantage of the variable memristance by programming memristor.When an input signal passes through the memristive synaptic circuit, the weighting operation is realized.Reported simulation results show the effectiveness of the memristive chaotic neural network, that is, it can separate stored patterns from superimposed input pattern and can deal with one-to-many associative memory and many-to-many associative memory perfectly.Furthermore, the memristive synaptic circuit makes the network structure of MCNN more simple and more powerful.At the same time, due to the simple structure two-terminal , small device size nano-scale of the memristor, the proposed MCNN model has a brighter future in physical implementation and holds potential to be implemented in VLSI chips.

Figure 1 :
Figure 1: The average firing rate of the chaotic neuron.

1 −Figure 2 :
Figure 2: Response characteristics diagram of 2.3 : a the bifurcation structure; b the Lyapunov exponent.

Figure 3 :
Figure 3: Characteristics of memristor.a Typical memristor i-v pinched-hysteresis loop; b memristor q-φ curve; c memristance versus state map; d memristance changes with a series of voltages pulses.

Figure 4 :
Figure 4: Structure of the memristive chaotic neural networks.

Figure 5 :
Figure 5: Simplified schematic diagram of the memristive synaptic circuit.

Figure 6 :
Figure 6: Three memristance states and the corresponding programming voltage pulses.

Figure 7 :
Figure 7: a -c The training set, d the input superimposed pattern.

Figure 8 :Figure 9 :
Figure 8: Output of a MCNN for separation of superimposed pattern.

Figure 12 :
Figure 12: Output of the proposed MCNN for many-to-many associative memory.