Adaptive Control Using Neural Networks and Approximate Models for Nonlinear Dynamic Systems

In this research, a comparative study of two recurrent neural networks, nonlinear autoregressive with exogenous input (NARX) neural network and nonlinear autoregressive moving average (NARMA-L2), and a feedforward neural network (FFNN) is performed for their ability to provide adaptive control of nonlinear systems. Three dynamical nonlinear systems of different complexity are considered. The aim of this work is to make the output of the plant follow the desired reference trajectory. The problem becomes more challenging when the dynamics of the plants are assumed to be unknown, and to tackle this problem, a multilayer neural network-based approximate model is set up which will work in parallel to the plant and the control scheme. The network parameters are updated using the dynamic backpropagation (BP) algorithm.


Introduction
Linear control methods are based on the existence of an analytical model of the system. However, most physical systems have nonlinearity, and their mathematical model is unknown or partially known and variable in time. So, conventional methods suffer some limitation in terms of stabilization and performance [1,2]. With the considerable development of artificial intelligence, which is found to be more suitable in handling such complex processes, the artificial neural network (ANN) is one of the powerful tools that has been recognized for tracking highly nonlinear dynamic and complex systems [2][3][4][5].
In the field of control engineering for a class of nonlinear discrete-time systems, a neural network has recently emerged as adjustable approximators capable of reproducing the complex behavior of nonlinear systems. Artificial neural networks have been effectively used as tracking controllers for unknown linear and nonlinear dynamic plants [6,7]. ANNs have been employed in various fields, like time series prediction, system identification and control, and function approx-imation [8]. It has been shown that ANNS can efficiently approximate dynamics without requiring detailed knowledge of the plant [8,9]. Another advantage of ANNs is their possibility of learning, which can reduce the human effort during the design of the controllers and allows discovering more effective control structures than those already known.
There are different types of ANNs proposed in the literature: feedforward neural networks (FFNNs), Kohonen selforganizing network, radial basis function (RBF), and recurrent neural network [10][11][12]. In fact, two classes of neural networks are the most popular in practical applications and have received considerable attention in the area of artificial neural networks in recent years: multilayer feedforward neural networks and recurrent networks. Multilayer feedforward neural networks have an ability to model any nonlinear function and have proven extremely successful in pattern recognition tasks [13,14] while recurrent networks have been used in associative memories as well as for the solution of optimization problems, and it constitutes also a powerful computational tool for sequence modeling and prediction. Both of these networks have common characteristics like parallelism in their operation, generalization, and learning capability, which make them suitable candidates for the control of nonlinear systems [2]; there are compelling reasons to view them in a unified fashion.
Various structures of neural networks have been proposed in the literature. In [2], the authors have applied a diagonal recurrent neural network (DRNN) as a controller for both single-input single-output (SISO) and multipleinput multiple-output (MIMO) plants. The responses of plants obtained with DRNN are compared with those obtained when a multilayer feedforward neural network is used as a controller. In [15], recurrent neural networks have been used for providing speed control to the nonlinear motor-drive system using a model-following control scheme. In [4], an adaptive controller for a class of unknown nonlinear discrete-time systems based on a multi-input fuzzy rules emulated network (MIFREN) is introduced; the neural network is assigned to identify the unknown system under control. The authors in [3] designed a nonlinear autoregressive moving average (NARMA-L2) controller, which is based on an adaptive neurofuzzy inference system. The NARMA-L2 controller for a single-link manipulator has also been used in [16,17]. A Lyapunov function-based neural network tracking (LNT) strategy for SISO discrete-time nonlinear dynamic systems is proposed. The proposed scheme uses two Lyapunov function neural networks operating as the controller and estimator. In [18], both the feedforward and recurrent neural network approaches are proposed, tested, and compared.
The main contribution of this paper is to propose a strong nonlinear adaptive controller of unknown nonlinear dynamical systems based on the approximate models. Another objective is to present a prescriptive method for the dynamic adjustment of the parameters based on backpropagation. The big advantage of the proposed control system is that it does not require previous knowledge of the model. Our ultimate goal is to determine the control input using only the values of the input and output.
In our research study, the strong learning capability of the dynamic neural network in identification and control is combined with the functionality of the approximate model controller structure to propose a novel online NN-based controller for nonlinear single-input single-output (SISO) dynamical systems. The main issues in the field of ANNbased control are the choice of neural network structure to be used as a controller. In our paper, an attempt has been made to compare the three types of neural networks: two recurrent neural networks, nonlinear autoregressive with exogenous input (NARX) neural network and nonlinear autoregressive and moving average (NARMA-L2), and a feedforward neural network (FFNN). The control system configuration employed in our paper consists of an ANNbased controller in cascade with the plant, and the training is performed online.
This study differentiates from the studies in the literature in terms of mathematically obtaining nonlinear SISO controller parameters. Thus, the main novelty of this paper is that the parameters of the nonlinear SISO controller can be identified as neural network expressions for nonlinear SISO systems. All neural network simulations are performed in a Matlab environment. The results indicate that the online NARMA-L2, NARX, and FFNN controllers attain good modeling and control performances.
The remainder of the paper is organized as follows: Section 2 includes a brief introduction about NARX, NARMA-L2, and FFNN models and their mathematical formulation. Section 3 includes the discussion on the adaptive control of nonlinear systems. Section 4 contains the simulation study. In Section 5, the conclusion of the paper is given.

Neural Network Approximate Models and Their Mathematical Formulation
Generally, there are many different neural networks (NN) of nonlinear models. In this research study, the recurrent and feedforward neural networks are presented. Since a lot of literature exists on NARX, NARMA-L2, and FFNN, in this section, a brief introduction regarding their mathematical formulation is given. Figures 1-3 show the structures of FFNN, NARX, and NARMA-L2 models, respectively.

Feedforward Neural Network
Model. In a feedforward neural network, the information moves only in one direction, from the input layer to the output layer, but not vice versa as presented in Figure 1 [19].  behavior of the NARX model may be written in the following form: Note that in Figure 2, we have assumed that the two delay line memories du and dy are both equal to the order of the plant or different with du ≤ dy, where f and g are a hyperbolic tangent and a linear activation function characterizing the hidden and output layers of the MLP, respectively, and N denotes the number of hidden layers.

Output neuron with linear activation function
Output layer Hidden layer Input layer

Input vector
Hidden layer with tangent hyperbolic activation function Multilayer perceptron    (Figure 3) is one of the most certified representations of general discrete-time nonlinear systems. This model representation is used in the form of the past, the current, and the future system parameters, as shown in [3] where yðkÞ and uðkÞ are the input and output of the system, respectively, the relative degree d represents the delay of the system from control effort u to the output y, and Φ½· is a nonlinear function. This model is not convenient for finding a control signal uðkÞ. To overcome this problem, an efficient method is proposed by Narendra and Mukhopadhyay by introducing approximation models. Two classes of the NARMA model have been proposed, namely, NARMA-L1 and NARMA-L2. It was found that the second class involving two subapproximation functions is more efficient and TDL Reference model
The NARMA-L2 is obtained by where f and g are, respectively, the activation functions of the hidden layers and the linear layers of the networks N 1 and N 2 . Equations (5) to (7) represent single-input single-output (SISO) systems. As it can be seen from equation (4), NARMA-L2 consists of two nonlinear functions, f 0 and g 0 , which can be approximated using two subnetworks (N 1 and N 2 ) as presented in equation (5). To design the neurocontroller, the number of delayed plant inputs and outputs is chosen based on a structural model. The size of the hidden layer is chosen such that the network can accurately approximate the nonlinearity of the system. There are two steps to construct the NARMA-L2 controller: identification and control.
The two subfunctions, N 1 and N 2 , are used in the identification phase as well as to compute a signal as follows: where y r ðk + dÞ is the reference signal to follow. This controller can be implemented using the model of the NARMA-L2 system previously identified. If the system is precisely approximated, the output of the system will be equal to the output of the reference model.

Controller Design
The control configuration based on the NN approximate model is shown in Figure 4. In this control scheme, a NN model is utilized to approximate the nonlinear function of the unmodeled dynamics. The system model is considered unknown. The use of neural networks is justified by this absence of a model. In this case, three approaches predominate. In the two approaches based on the FFNN and NARX models, a single network is used for system control. The third approach based on the NARMA-L2 model uses two networks to deduce the controller adaptation law.
3.1. Learning Algorithm. The process of training consists in modifying the weights in an organized way using an appropriate algorithm. During the training process, a specified Modelling and Simulation in Engineering number of inputs and their desired output are introduced in the network. Then, the weights are tuned so that the neural network produces an output close to the target values. The fundamental training algorithm for multilayer networks is the backpropagation (BP) algorithm [20]. This algorithm is of iterative type, and it is based on the minimization of a

Modelling and Simulation in Engineering
sum-squared error (MSE) utilizing the optimization gradient descent method. The MSE is used as the cost function which is a function of error, which is defined as follows: where y r ðkÞ and yðkÞ are the desired output of the network and the actual response of the network on the given input pattern uðkÞ, respectively, and N is the dimension of the training set. The modification of the weights of the jth neuron in the μth layer is performed according to the formula: where Thus, each element in W μ ðkÞ and b μ ðkÞ can be updated as The error δ is defined as follows: where f ðμÞ ðxÞ = tan hðxÞ = ðe x − e −x Þ/ðe x + e −x Þ and f μ+1 ðxÞ = x.

Results and Discussion
Simulation studies have been carried out using the Matlab software environment to verify the performance of the proposed methods. Three different nonlinear dynamical systems are presented to evaluate the performance of the NN controllers. Two hidden layers are used with H1NN = 20 and H2N N = 10 (thus, 20 neurons are present in the first and 10 neurons in the second hidden layers).

Example 1.
Consider the nonlinear discrete time which is described by the third-order difference equation [19]: The system can be reformulated as A reference model is described by the third-order difference equation: where rðkÞ is a bounded reference input. The function h is considered to be unknown; it can be estimated using neural network N h . The control input to the plant at any instant k is computed using Nhð−Þ in place of h.
If yðk + 1Þ = yrðk + 1Þ, then, Modelling and Simulation in Engineering whereas for NARMA-L2 the control signal is generated based on equation (7): The simulation reported in Figures 5-7 indicates stable and efficient online control. The improvements in the responses, when the neural networks in the approximate model are used to generate the control input to the plant, are evident from the figure. The outputs of the controlled plant and the reference model are shown and indicate that the output error is almost zero. All three neural networks FFNN-, NARX-, and NARMA-L2-based controllers provided satisfactory results.
The performance index of the different controllers with various neural network models is given in Table 1. We can notice that all the designed controllers give better performance. Nevertheless, the performance of the FFNN and NARX indicated better attainment regarding the performance indices (MSE).

Example 2.
Consider the nonlinear discrete-time system [1,17]: where yðkÞ stands for the system output and uðkÞ is the control input at time index k. The output yðk + 1Þ is required to track the desired trajectory y r ðk + 1Þ given in The order of the plant is n = 1, so the inputs for the FFNN controller are y r ðkÞ and yðkÞÞ and the inputs for the NARMA-L2 controller are yðkÞ and uðkÞÞ, and NARX models are y * ðkÞ, yðkÞ,and uðkÞÞ.
The control results for different epochs and learning rates along with control error are shown in Figure 8. As we can see from the results, the network is unstable using n = m = 1 in the delayed input/output (not enough information).

Modelling and Simulation in Engineering
The model of discrete-time plants introduced here can be also described by the following nonlinear difference equations: The same proposed scheme in Figure 4 is considered. To approximate the unknown h, an FNN-based identification model is used: The control action is required to get y r ðkÞ = yðkÞ.
In this example, performance comparisons of the BP-NNs are carried out with the three approximate models, FFNN, NARX, and NARMA-L2. The results are shown in Figures 9-11. Also, notice that after sufficient online training of the plant, it has also converged to the desired response. The result during the training is presented in the figures. It can be seen from these figures that the control error (ec) reduces quickly to zero in the case of the NARX and NARMA-L2 controllers as compared to the control error obtained with FFNN. This suggests the strong learning ability of recurrent NN models over FFNN in this example.
The numerical parameters used in the second system are presented in Table 2, as well as the mean square error (MSE) obtained with different controllers.

Example 3.
The second-order differential equation which describes the dynamics of a single-link robotic manipulator is given as The angular position of the robotic manipulator arm is represented by yðtÞ. The other parameters include link length which is denoted by l. Also, the term DðdyðtÞ/dtÞ quantifies the viscous friction torque, and g = 9:8 m/s 2 is the acceleration due to gravity. For simplicity, the values of m = l = D = 1 are used in this paper. Further, u ðtÞ is the control torque exerted on the robotic arm. The corresponding state space representation of the above differential equation is given by The difference equation of the one-link robotic manipulator with a sampling period is given by where the nonlinear function h is The desired reference model dynamics is given by the following difference equation: where rðkÞ = sin ðkTÞ is the externally applied input. Before control action, the sampling period T should be chosen with care. It is taken to be T = 0:4. In this part, the    control is to make the robotic link output follow the reference trajectory signal. The response of the plant without control is shown in Figure 12. Clearly, the plant's output is not following the reference input. So, the control scheme is set up and the training is continued for 100 epochs, and each epoch contains 100 training samples.
In the simulation studies, N 1 and N 2 were multilayer neural networks chosen with three-layer networks with p inputs, 20 neurons in the first hidden layer, 10 neurons in the second hidden layer, and one output neuron.
From equation (25), the control input uðkÞ can be computed from acknowledging yðk + 1Þ and its past values as r k ð Þ = sin 2πk 25 + sin 2πk 10 : The mean square error (MSE) obtained with different controllers for example 3 is presented in Table 3. From the table, it can be seen that the MSE obtained with FFNN, NARX, and NARMA-L2 is minimum as compared to the MSE obtained with the RBFN-and DRNN-based controllers.
In this study, all the three methods provide simultaneous structure and parameter learning within the approximate models. The learning rate, η, is taken to be 0.01. From Figures 13-15, it can be seen that the response of the robotic link is successfully controlled.

Conclusions
This paper describes the application of NN models as a controller. Both feedforward and two recurrent NARX and NARMA-L2 predictors are proposed, tested, and compared. The method for the adjustment of parameters in generalized neural networks is treated, and the concept of dynamic back propagation is introduced. The training of all the three neural networks is done online. Simulation and comparative studies demonstrate the superior performance of the proposed approaches. It can be concluded that the ANNs can be successfully utilized for controlling different nonlinear dynamic systems. In addition, future works will explore the adaptive neural network control for nonlinear MIMO systems under disturbances using the hybrid learning method.

Data Availability
No data were used to support this study.

Conflicts of Interest
The authors declare that they have no conflicts of interest.