Memristive Chebyshev Neural Network and Its Applications in Function Approximation

Anovel Chebyshev neural network combinedwithmemristors is proposed to perform the function approximation.The relationship between memristive conductance and weight update is derived, and the model of a single-input memristive Chebyshev neural network is established. Corresponding BP algorithm and deriving algorithm are introduced to the memristive Chebyshev neural networks. Their advantages include less model complexity, easy convergence of the algorithm, and easy circuit implementation. Through the MATLAB simulation results, we verify the feasibility and effectiveness of the memristive Chebyshev neural networks.


Introduction
Many researchers have further studied the approximation capacity of neural networks.Research indicates that there are some drawbacks in Back Propagation (BP) neural networks, such as slow convergence, getting easily into a local minimum, and difficulties in determining the numbers of hidden neurons.The Radical Basis Function (RBF) neural networks are superior to BP neural networks in approximation capacity because their learning rate is fast, but the lack of theoretical guidance in determining the central position of basis function makes us set the central position based on experience.We cannot ensure whether the performance of networks is the best or not.Especially, there is a dilemma between approximation accuracy and networks complexity when we use the aforementioned two neural networks to approximate the nonlinear function.However, the Chebyshev neural networks can solve the bottleneck; their hidden layer neuron activation functions are a group of Chebyshev orthogonal polynomials.Nowadays, Chebyshev neural networks are widely used in complex nonlinear systems [1], chaos systems [2], and discrete-time nonlinear systems [3].Their learning rate and approximation accuracy are better than traditional neural networks, and they can quickly determine the numbers of hidden neurons.Meanwhile, Chebyshev neural networks are widely used in aerospace [4,5] and chemistry areas [6].
The nanoscale memristor has the potential of information storage because of the nonvolatility with respect to long periods of power-down, so it can be used as synapse in the neural networks.Many researchers in the field of memristors have suggested that this device has high potential for implementing artificial synapses.Afifi et al. studied the realization of STDP learning rules based on the pulsing neuromorphic networks with memristor cross array [7].Hu et al. built a novel chaotic neural work with memristor to implement associative memory [8].Wang et al. proposed a PID controller based on memristive CMAC network [9].Pershin and di Ventra realized the associative memory based on memristive neural networks [10].Sharifi and Banadaki applied the memristors as memory units to the nonvolatile RAMs and as synapse to the artificial neural networks [11].Chabi and Klein put forward a neural network with high fault tolerance based on a structure of crossbar switches [12].Cantley et al. built neural networks with synapses made up from amorphous silicon thin film transistor and memristor, which realized the Hebb learning rules [13].Gao et al. designed cellular neural network, which is applied to image denoising and edge detection [14].Kim et al. changed the synapse weight of artificial neural networks with a pulsebased programmable memristor [15].
The memristive Chebyshev neural networks have the following advantages: (a) single-input memristive Chebyshev neural networks can realize any nonlinear function approximation by only three layers; (b) the number of hidden neurons in memristive Chebyshev neural networks is significantly smaller than that in traditional BP neural networks; (c) we only need to adjust the weight from the hidden layer to the output layer, which can greatly help in quickening the convergence of the algorithm; (d) as memristive synapse is small and passive, the hardware circuits of the memristive Chebyshev neural networks can be implemented by the VLSI circuits easily.
In this paper, we establish the correspondence between memristor conductance and synapse weight through theoretical analysis and MATLAB simulations.The nanoscale memristor is the synapse in Chebyshev neural networks to realize function approximation by memristive BP algorithm and memristive deriving algorithm, respectively.

Memristive Synapse
The physical model of the memristor [16] is shown in Figure 1.W(t) and  represent the thickness of the doped layer and total oxide films, respectively. ON is the value of the memristor when W(t) is equal to , and  OFF is the value of the memristor when W(t) is equal to zero.M(0) is the initial value of the memristor.
According to the mathematical model of HP memristor, and the flux-controlled model of memristor, we can obtain the following equation [17]: where .  ] is the average drift rate of oxygen vacancies, so we can obtain the following equation from (1): ( The derivate of equation ( 2) is (3) The physical model of the memristor of HP [16].
Using calculus, dG(t) is approximately equal to ΔG when Δ approaches to zero, and then we can obtain the following equation: where d/dt = v(t); the relationship curve between memristive conductance change and voltage is shown in Figure 2. It can be seen that memristive conductance change Δ increases along with the increasing of applied voltage V.If the learning error  of neural networks is regarded as voltage v, the memristive conductance change can reasonably be described as synapse weight update whose correspondence can be built.

Modeling of the Memristive Chebyshev Neural Networks
The nth power orthogonal polynomials   (x) which are related to the weight function () = 1/ √ 1 −  2 are called Chebyshev polynomials of the first kind [18].Based on definition, if the polynomial system {  ()} (where  = 0, 1, 2, . ..) and the weight function satisfy the following relationship: the orthogonal polynomial system {  ()} can be structured in the interval of [−1, 1].
In this paper, a model (see Figure 3) of the single-input memristive Chebyshev neural networks is proposed, which consists of the input layer, hidden layer, and output layer.The weights from input layer to hidden layer are set to 1, and the memristor conductance stores the weights of hidden layer to output layer.The thresholds of all neurons are set to 0. The hidden layer has  neurons, and their transfer functions are a group of Chebyshev orthogonal polynomial functions   (x).

Realization of Function Approximation Algorithm
Set memristive Chebyshev neural networks model as follows: input layer: o = x; input of hidden layer neuron: net  = o; output of hidden layer neuron:   =   (net  ); output layer: Samples (  , f (  )),  = 1, 2, . . .,  (s is the number of the samples), are assumed as inputs, and the errors of network output  and the target value f (x) are   = f (  ) −  .
Network training index is 4.1.Memristive BP Algorithm.In order to establish the relationship between the memristor and the synapse, we propose the learning rule for memristive synaptic weight update.According to (4), we assume the memristive BP algorithm as follows: where  = 0, 1, 2, . . .,  − 1, Δ =  is the learning rate, K is the number of learning, and Γ is the integral of the error e.The learning algorithm is described as follows.

Memristive Deriving Algorithm.
The basic principles of the memristive deriving algorithm are as follows.Take any less  hidden neurons as the initial cells.After training  times, the error e is no longer changed; that is, E(K) = E(−1) >  ( is a pretraining given precision index).The network derives automatically by adding a hidden neuron ( ← +1).
The previous process is repeated until E(K) ≤  when the hidden neurons stop deriving and learning.Until reaching target accuracy, the neural network topology is automatically generated.
The specific algorithm can be described as follows.
Step 2. Input of hidden layer neuron is Calculate  Step 3. Weight update rule is Step 4. Calculate If E(K) ≤ , stop derivative and learning which illustrate that neural network topology is automatically generated.Otherwise, go to Step 5.
Step 5.If E (K) = E − 1), the training error of the network has no longer reduced.We should increase the number of hidden neurons in order to improve network performance.Namely,  ← +1,  ← +1, and jump to Step 2 to continue learning.

Memristive Deriving Algorithm Realizes Function Approximation.
We adopt the 1 × 2 × 1 initial neural network, with  = 1.0 × 10 −5 and learning rate 0.02, and the experimental result is shown in Figure 5.The final network structure is 1 × 25 × 1, and the time of learning is 420. Figure 5  shows the approximation curve of the actual output signal relative to the teacher signal.
Compared with the memristive BP algorithm, the memristive deriving algorithm has less learning times (735 − 420 = 315) for its neural network is automatically generated.

Conclusions
The HP memristor model is analyzed, and the correspondence between the memristive conductance change and the synapse weight update in Chebyshev neural networks is established.Two learning algorithms of memristive synapse weight update are proposed.The nonlinear function approximation is implemented by memristive BP algorithm and memristive deriving algorithm, respectively.On the one hand, the memristive BP networks have fixed structure which is easily implemented.On the other hand, the memristive derivative networks have dynamic structure reaching the optimum solution.Memristive Chebyshev neural networks are hopeful to be used in aspects of pattern recognition and data compression.

Figure 2 :
Figure 2: The relation curve of the change of memristive conductance and voltage.
Approximation.The 1 × 8 × 1 network structure is shown in Figure 3.The learning rate  = 0.02.The number of samples s = 200.The number of learning is 1000.The target mean square error E = 1.0 × 10 −5 .The experimental result is shown in Figure 4.The relationship between training error and the number of learning is shown in Figure 4(a).The number of final learning is 735 when the minimum training error meets the requirements of function approximation.The memristive synaptic weights are constantly updated in the learning process as shown in Figure 4(b).Figures 4(c) and 4(d) describe the final memristive weights and the approximation curve of actual output signal relative to the teacher signal, respectively.
(a)  shows the final memristive synaptic weights in hidden layer.Figure5(b) Relationship between training error and the number of learning
The final memristive weights in the hidden layer