Controller Design Based on Echo State Network with Delay Output for Nonlinear System

. For the nonlinear systems with delay output, the control performance of the system is aﬀected by the previous output of the system, such as crawling robots of photovoltaic panels. In this paper, an improved controller design method based on echo state network with delay output (DO-ESN) is proposed for designing the controller of a class of nonlinear system. According to the internal characteristics of DO-ESN, the DO-ESN can match the system characteristics of nonlinear systems with delay output, such that the proposed controller can quickly meet the control performance of the nonlinear system. In order to ensure the stability of the controller, a suﬃcient condition is given for the echo state property of DO-ESN. Finally, a simulation example is used to illustrate the eﬀectiveness of the proposed method.


Introduction
For the nonlinear system, the proportional-integral-derivative (PID) method is a very effective control method, which has been widely applied in many fields, for example, load frequency control [1], automatic voltage regulator [2], chemical engineering [3], vehicle dynamic [4], hydrogen energy [5], and others [6]. However, the setting problem of PID controller parameters is always the focus in the research of academic and industry community. In recent years, based on neural network, many different controller design methods have been presented, for example, BPNN [7], RBFNN [8,9], single neuron adaptive neural networks (SNANN) [10], adaptive neural networks [11], and artificial intelligence algorithm [12]. For these design methods based on artificial neural networks, some appropriate PID parameters can be obtained to meet the control accuracy requirement in some industrial applications.
However, there are still some problems for these design methods, for example, through using BPNN, the obtained PID controller parameters are easily trapped in local minima. Because of the randomness of the center vector, it is difficult to obtain the appropriate PID controller parameters by using RBFNN in a short time. rough analyzing these design methods based on artificial neural networks, one can summarize these drawbacks including training weights, local minima, and slow convergence. erefore, an improved neural network should be introduced into the controller design method to solve the above shortcomings.
Echo state network (ESN) [13,14] uses a dynamical reservoir with many random connected neurons to replace the hidden layer of recurrent neural network (RNN) [15][16][17][18][19][20]. Compared with the traditional RNN, the advantages of ESN are reflected in the weight selection and weight learning of network, i.e., only the output weight needs to be learned. erefore, ESN not only has the network structure of traditional RNN but also has the characteristics of deep learning, such that ESN can be applied in many fields, for example, time-series prediction [21][22][23][24], filtering or control [25][26][27][28], dynamic pattern recognition [29][30][31], optimization [32], system identification [31,33,34], and big data application [35,36]. us, comparing with the existing controller design methods based on neural network, ESN can avoid lots of adjusting parameters and the limitation of calculation. Because the reservoir of ESN has the memory ability, the previous calculation results can be used for the next parameter calculation. In addition, ESN can also deepen the calculation of input information to improve the convergence speed.
Combining with the advantages of ESN, some improved methods have been applied in many control fields. For example, in [26], a PID adaptive controller based on RLSESN is proposed for realizing high-accurate tracking of the rehabilitation robotic hand. In [27], ESNBIMC is proposed for the pneumatic muscle system. e ESNBIMC is regarded as a replacement of the conventional PID controller which is used to adjust the input of the controlled plant. In [28], through using the approximation capability of ESN, the ESN is regarded as a controller which is applied for nonlinear dynamical systems. However, for a class of nonlinear systems with delay output, the current moment output of the system is affected by some previous moment output of the system, such that these existing control methods based on echo state network cannot well match the characteristics of the system. us, for the nonlinear systems with delay output, how to build a suitable echo state network to meet the characteristics of the system is an interesting problem.
us, in this paper, an improved echo state network with delay output (DO-ESN) is proposed. Based on the DO-ESN, an improved controller design method is given for a class of nonlinear systems with delay output, such that the output of the DO-ESN controller can quickly meet the control performance of the nonlinear system. In order to ensure the stability of the controller, the echo state property of DO-ESN should be first guaranteed, and thus, a sufficient condition is given for the echo state property of DO-ESN. e main work of this paper is given as follows: (i) An improved controller design method based on DO-ESN is given for a class of nonlinear systems with delay output, such that the control performance of this nonlinear system can be quickly satisfied (ii) A sufficient condition is given for the echo state property of DO-ESN, and the proposed controller can be stably applied to nonlinear systems with delay output

Echo State Network with Delay Output
ESN is a class of recurrent neural networks, whose structure is shown in Figure 1. From Figure 1, the ESN comprises a K-dimensional input layer, a N-dimensional reservoir, and a L-dimensional output layer. Let u � u(n), x � x(n), and y � y(n) denote the external input vector, the reservoir state, and the output vector, respectively. W in ∈ R N×K denotes the input weight matrix, W ∈ R N×N denotes the reservoir weight matrix, W fb ∈ R N×L denotes the output feedback weight matrix and W out ∈ R L×(K+N) denotes the output weight matrix. For the standard ESN, the reservoir state updated equation is given as follows [14]: where f(·) and g(·) denote the activation function of reservoir state and network output, respectively.
Consider the characteristics of the nonlinear system with delay output, the previous moment output of ESN can be introduced into the reservoir state equation, and thus an improved echo state network with delay output (DO-ESN) is proposed. e reservoir state updated equation of DO-ESN is given as follows: where τ denote the index of delay output.

Remark 1.
For the DO-ESN, the current reservoir state and the current output are affected by the previous output. e current reservoir state can use the previous system characteristic, and the current output can use the previous output characteristic. erefore, the DO-ESN can better match the characteristics of the nonlinear system with delayed output.

Controller Based on DO-ESN.
Considering a class of nonlinear system with delay output, where τ 1 and τ 2 denote the delay of y(n) and u c (n), respectively. f s denotes the nonlinear polynomial. According to the learning ability and approximation ability of DO-ESN in control and optimization, the DO-ESN can be embedded into the controller to control the nonlinear system. e design controller method based on the DO-ESN is illustrated in Figure 2.
From Figure 2, e s (n) denotes the system output error, y s (n) denotes the actual system output, and r(n) denotes the desired system output.

Input
Output Reservoir Figure 1: Structure of ESN.

Stability Analysis of Controller Based on DO-ESN.
In order to guarantee the stability and convergence of the controller model, the echo state property of DO-ESN should be ensured. us, in the following, a sufficient condition will be provided to guarantee the echo state property of DO-ESN.
Because the W out (n) can be written as follows: e output of DO-ESN is modified as follows: us, the reservoir state is modified as follows: where (2) and (3)), if the following conditions are satisfied

Theorem 1. For a DO-ESN model (equations
(1) f is a Lipschitz continuous function, and the Lipschitz constant is less than 1 max and δ * fbmax are the maximal singular value of W * and W * fb W out x , respectively); then, the DO-ESN model has the echo state property in each iteration Proof. For x(n + 1) and x ′ (n + 1), we have For the term y(n − τ) − y ′ (n − τ) of equation (8), we have Similarly, we also have Substituting equations (10) and (11) into equation (9), we have Complexity 3 Substituting equation (12) into equation (8), we have If en, the DO-ESN has the echo state property in each iteration.

Remark 2.
For the proposed controller design method based on DO-ESN, the stability of the controller should be guaranteed, and then, a sufficient condition need be given for the echo state property of DO-ESN. us, in this section, the proposed controller design method can be stably applied to nonlinear systems with delay output.

Learning the Output Weights of DO-ESN.
e purpose of designing the controller is to minimize the system output error. e expression of the system output error e s (n) is given as follows: For the output weight learning of DO-ESN, the partial derivative ze s (n)/zW out (n) need be computed. ze s (n)/ zW out (n) is formulated as follows: en, the output weights W out (n + 1) is updated as follows: where β denotes the learning rate.
Remark 3. In this paper, we usually give a definite value for the output weight learning rate. Since the learning rate is also affected by the reservoir parameters of DO-ESN, for different nonlinear systems, the learning rate can be given according to the changes of the reservoir state in the process of training network.

Simulation Examples
In this section, a simulation example is selected to show the performance of the proposed control method. For different output error accuracy (5%, 2%, 0.1%, or E-10), several neural networks methods (e.g., BPNN [7], RBFNN [9], and ESN [15]) are selected to illustrate the performance of the proposed design method. Considering the following nonlinear system with time delay, e desired output r(n) of system is where t s � 0.001. According to the characteristic of the nonlinear system, the reservoir size of DO-ESN is given as N � 25. rough equation (17) with β � 0.0002, the final reservoir parameters of DO-ESN are obtained as follows: e simulation results of these methods (BPNN [7], RBFNN [9], ESN [15], and DO-ESN) are shown in Table 1. e curves of system output and system output error are shown in Figures 3 and 4, respectively.
From Table 1, we can see that when the output value of the system reaches the expected fluctuation range (5%-2%), the DO-ESN needs less iteration step. For example, when the error precision is 0.025(5%), the iteration step of DO-ESN is 22, and the iteration step of those methods [7,9,15,21] are 60, 145, 40, and 105, respectively. When the error precision is 0.01 (2%), the iteration of DO-ESN is 34, and the iteration 4 Complexity of other methods [7,9,15,21] is 74, 212, 62, and 123, respectively. When the error precision is 0.0005(0.1%), the iteration of DO-ESN is 56, and the iteration of other methods [7,9,15,21] is 125, 462, 96, and 245, respectively. From Figures 3 and 4, we can see that, for the same error precision, the iteration step of DO-ESN is far less than that of those methods in [7,9,15].

Discussion
In this paper, an improved controller design method based on DO-ESN is proposed for a class of nonlinear system with delay output. According to the internal characteristics of DO-ESN, the DO-ESN can match the system characteristics of nonlinear systems with delay output, such that the DO-ESN controller can quickly meet the control performance of the nonlinear system. Meanwhile, a sufficient condition for the stability of the DO-ESN controller is given, such that the controller can be stably applied to the given controlled plant.
Finally, the simulation result shows the effectiveness of the proposed method.

Data Availability
e data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
e authors declare that they have no conflicts of interest.