Adaptive Neural Network Motion Control of Manipulators with Experimental Evaluations

A nonlinear proportional-derivative controller plus adaptive neuronal network compensation is proposed. With the aim of estimating the desired torque, a two-layer neural network is used. Then, adaptation laws for the neural network weights are derived. Asymptotic convergence of the position and velocity tracking errors is proven, while the neural network weights are shown to be uniformly bounded. The proposed scheme has been experimentally validated in real time. These experimental evaluations were carried in two different mechanical systems: a horizontal two degrees-of-freedom robot and a vertical one degree-of-freedom arm which is affected by the gravitational force. In each one of the two experimental set-ups, the proposed scheme was implemented without and with adaptive neural network compensation. Experimental results confirmed the tracking accuracy of the proposed adaptive neural network-based controller.


Introduction
Robust control consists in designing control strategies by using little information of the system model and considering that the system may be affected by bounded disturbances. Robust controllers can be designed to satisfy either the regulation goal or the trajectory tracking objective. Thus, in the last years there has been mathematical and practical interest in studying robust control architectures. In this class of controllers, adaptive neural networks have been used in the design of robust controllers for electromechanical systems.
Neural networks can be used in the control of unknown systems without requirements for linearity in the system parameters. Neural networks exhibit the universal approximation property which allows approximating unknown linear and nonlinear functions [1].
In the following, we provide a literature review on application of neural networks for robot motion control. Selmic and Lewis [2] proposed a dynamical inversion compensation scheme by using a backstepping technique with neural networks, which was applied to mechanical systems. Kwan et al. [3] proposed a class of neural network robust controllers showing global asymptotic stability of tracking errors and boundedness of neural network weights. Lewis et al. [1] provided in manner of survey a study of the application of neural networks in the compensation of actuator nonlinearities. In [4] a robust controller with a nonlinear two-layer neural network structure was proposed for control of four-axis SCARA robot manipulator. The error trajectories are proven to be uniformly ultimately bounded. Yu and Li [5] propose a PD-type controller plus neural network compensation of the uncertainties and velocity estimation is achieved by using a high-gain observer. Stability is proven using Lyapunov-based analysis. Wang et al. [6] proposed a neural network-based motion controller in task space. The controller is addressed as two-loop cascade control scheme. The inner loop implements a velocity servo loop at the robot joint level using a radial basis function network with a proportional-integral controller. In [7] an adaptive neural network algorithm is developed for rigid-link electrically driven robot systems. The controller is developed in a constructive form and a rigorous stability analysis is also provided. In Moreno-Armendáriz et al. [8], an indirect adaptive control using hierarchical fuzzy CMAC neuronal 2 The Scientific World Journal network for the ball and plate system was introduced. The proposed controller was validated by means of numerical simulations and experiments.
Fateh and Alavi [9] introduced a new scheme for the impedance control of an active suspension system. The control was achieved through two interior loops which are force control of the actuator by feedback linearization and fuzzy control loop. The work proposed by Liu et al. in [10,11] addressed the problem of control of manipulators with saturated input. In particular, new controllers based on fuzzy self-tuning of the proportional and derivative gains were proposed. Although the controllers in [9][10][11] are based in fuzzy logic, it has been proven in [12] that fuzzy logic systems and feedforward neural networks are equivalent in essence.
In Sun et al. [13], a robust tracking control for robot manipulators in the presence of uncertainties and disturbances was proposed. A neural network-based sliding mode adaptive control was designed to ensure trajectory tracking by the robot manipulator. In Hernandez et al. [14], a neural PD with a second order sliding mode compensation technique was introduced. The scheme is able to guarantee asymptotic convergence of the error trajectories. In [15], in order to deal with actuator and model nonlinearities, a neural networkbased controller was addressed. The scheme was experimentally tested in real time showing the advantages of the neural network. In [16], neural network contouring control using a Zebra-Zero robotic manipulator was presented. The paper in [17] used two modified optimal controllers based on neural networks. The closed-loop system trajectories were studied in a rigorous form although validation was presented with simulations. Other novel approaches of neural networks are presented in [18,19].
Experimental evidence that neural networks are efficient in the control of a mechanical systems has been provided in [8,15], for example. However, existing literature reveals a gap in the experimental evaluation of new controllers since most of the published works only consider numerical simulations to assess the performance of the proposed controllers.
In the present work a different approach is taken. In the theoretical part of our work, a neural network is used to approach the desired torque as a function of the desired joint position trajectory. The inner and outer weights of the neural network are adapted on-line by using an update law coming from a Lyapunov-like analysis of the closed-loop system trajectories. In addition, an extensive real-time implementation study in two different experimental set-ups has also been carried out. We prove by means of the experimental tests that the neural network compensation is really effective to reduce the joint tracking error. The real-time experiments show that an excellent tracking accuracy can be obtained by using adaptive neural network compensation plus a "small amount" of nonlinear PD control compensation. In summary, the contribution of this paper is twofold: (1) a new nonlinear PD controller plus adaptive neural network compensation, (2) a real-time experimental study in two different experimental set-ups.
Our approach is based on the adaptation of the weights of a neural network that only depends on the desired signals of joint position, velocity, and acceleration. In other words, the neural network used in the controller approaches the desired torque. In addition to the adaptive neural network feedforward compensation, the proposed controller is equipped with nonlinear PD terms, which are motivated by the convergence analysis. We prove that by using the proposed controller; the position and velocity error trajectories converge to zero, while the adapted neural network signals remain uniformly bounded. It is noteworthy that the proposed neural network controller resembles the PD control with feedforward compensation for robot manipulators, whose global asymptotical convergence proof was reported in [20,21]. The proposed controller is tested in real time in a horizontal two-degree-of-freedom direct-driven arm and in a vertical single-link arm, which is affected by the gravitational force. In both experimental set-ups, the new scheme is implemented with and without neural network adaptation. The experimental results show that the tracking performance is drastically improved by using the new controller with the adaptive neural network feedforward compensation.
The difference of this document with respect to our previous work in [22] is that here the study of closedloop trajectories is rigorously presented and experimental validation of the proposed scheme is evaluated in a detailed form.
The present document is organized as follows. Mathematical preliminaries, the robot model, and the control goal are given in Section 2. The proposed controller is discussed in Section 3. Section 4 is devoted to the real-time experimental result, while some concluding observations are given in Section 5.

Mathematical Preliminaries, Robot Model, and Control Goal
The notations min { } and max { } denote the minimum and maximum eigenvalues of a symmetric positive definite matrix ∈ R × , respectively. ‖ x ‖= √ x x stands for the norm of Given ∈ R × the Frobenius norm is defined [1,23] by Other properties are 2.1. Properties on Hyperbolic Functions. Some properties on hyperbolic functions will be used. See [24,25], where the cited properties are used in the design and analysis of controllers The Scientific World Journal 3 for mechanical systems. The tangent hyperbolic function is defined as where ∈ R. The tangent hyperbolic function can be arranged in a vector in the following way: and the following properties are accomplished by tanh(z).

Neural Networks.
Let us recall the universal approximation property of the neural networks [1,3]. A function f(x) : R +1 → R can be approximated by where x ∈ R +1 is the vector of input signals to the neural network, ∈ R ( +1)× and ∈ R × are the input and output ideal weights, respectively, is the number of neurons in the hidden layer, + 1 is the number of input signals to the neural network, ∈ R is the activation function in the hidden layer, and ∈ R is the approximation error with where > 0.
The output of an activation function, : R → R, = 1, . . . , , is used to define the output signal of a neuron from a modified combination of its input signals by compressing the signal. The function is usually between the values 0 ≤ ≤ 1 or valued such that −1 ≤ ≤ 1 is satisfied.
In this paper, we have used as activation function the hyperbolic tangent function. Therefore, by defining z = x, 2.3. Robot Dynamics. The dynamics in joint space of a serialchain -link robot manipulator considering the presence of friction at the robot joints can be written as [26,27] (q)q + (q,q )q + g (q) + Vq = , where q ∈ R is a vector of joint positions, (q) ∈ R × is the symmetric positive definite inertia matrix, (q,q )q ∈ R is the vector of centripetal and Coriolis torques, g(q) ∈ R is the vector of gravitational torques, V ∈ R × is a diagonal positive definite matrix containing the viscous friction coefficients of each joint, and ∈ R is the vector of torques input. The dynamics of -link robotic manipulator expressed in (13) has the following properties, which hold for rigid-link revolute joint manipulators [26][27][28].
Property 2. Assuming that the robot has revolute joints, the vector (q, x)y satisfies the bound where 1 > 0.
Property 3. Assume that the centrifugal and Coriolis torque matrix (q,q ) is computed by means of the so-called Christoffel symbols of the first kind. Then, Besides,̇( Property 4. The so-called residual dynamics [29,30] is defined by where e will be defined later and q ( ) is the desired joint position trajectory assumed to be bounded together with its 4 The Scientific World Journal first and second time derivative. The residual dynamics (18) satisfies the following inequality [27]: where ℎ1 and k ℎ1 are sufficiently large strictly positive constants that depend on the robot model parameters and > 0.

Control Goal.
Let us define the joint position tracking error as where q ( ) ∈ R denotes the desired joint position trajectory. The estimated weight errors are defined as where ∈ R [ +1]× are the ideal input weights,̂∈ R [ +1]× is an estimation of the input weights, ∈ R × are the ideal output weights, and̂∈ R × is an estimation of the output weights.
The desired time-varying trajectory q ( ) is assumed to be three times differentiable and bounded for all time ≥ 0 in the sense where 1 , 2 , and 3 denote known positive constants.
The control problem consists in designing a controller ( ) and update laws for the estimated weightŝ( ) and̂( ) such that the signals e( ),ė ( ),̃( ), and̃( ) are uniformly bounded. In addition, the limit should be satisfied.

Proposed Adaptive Neuronal Network Compensation Controller
The proposed controller has a structure of a nonlinear PD controller plus adaptive neural network feedforward compensation. The design of the controller departs from the assumption that the desired torque can be approached by a neural network and then it can be estimated on-line by means of proper adaptation laws for the input and output neural network weights.

Proposed Scheme.
The development of the proposed approach is presented in a constructive form. First, let us consider the robot dynamics (13) evaluated along the desired position q ∈ R such that the desired torque ∈ R can be founded by by combining (13) and (24) and using the tracking error defined in (20) the following equation is obtained: where h( , e,ė ) ∈ R is the so-called residual dynamics [29,30], defined by (18). By using the universal approximation property of the neural networks in (10), the desired torque in (24) can be approached as where is the vector of input signals to the neural network. Notice that in this case = 3 . Now, we are in position to introduce the following tracking controller: where , , and Δ are diagonal positive definite matrices, is a positive scalar,̂is the estimated input weight,̂is the estimated output weight, with The proposed update laws for the estimated input and output weights, denoted aŝand̂, respectively, arė where ∈ R [ +1]×[ +1] and ∈ R × are positive definite matrices,̂= (̂x ) and witĥx .

Closed-Loop System
Derivation. By substituting (26) and (28) in (25), the equation The Scientific World Journal 5 is obtained. The weight error is defined as in (21) and Multilayered neural networks are nonlinear in the weights and Taylor's series can be used to approximate the activation function . Thus, where (̃x ) 2 represents the higher order terms. Approximation of activation function via Taylor's series has been used in, for example, [1,3]. Therefore, substituting (21) and (37) in (35) and simplifying, we obtain the following expression: where = (q), h = h(e,ė ), = (q,q ), and Finally, the dynamics of [eė ] ∈ R 2 is given by By using the definition of the weight errors (21), we can rewrite the proposed update laws (32)-(33) as The overall closed-loop system is given by (40) and (41).

Convergence Analysis.
The assumption with > 0 and ( ) defined in (39), will be used in the next. Concerning the trajectories of the closed-loop system (40)-(32), we have the following result.
which is positive definite in terms of the state space of the closed-loop system (40)-(32). By using property (9), a lower bound on ( , e,ė ) can be computed as follows: ] ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ If is positive definite, then the function ( , e,ė ) is globally positive definite and radially unbounded. By using Sylvester's Theorem, the sufficient and necessary condition for to be positive definite is Next step in the proof is to compute the time derivative of ( , e,ė ) along the closed-loop system trajectories (40) and (32). Thus, we have thaṫ An upper bound on each term oḟ( , e,ė ) is computed as follows: where the property of the residual dynamics h in (19), assumption (42), property (8), property (15), and the fact that ‖q ‖≤ 2 + ‖ė ‖, with 2 defined in (22), were used. With the computed bounds on each term oḟ( , e,ė ), we can writė ( , e,ė ) ≤ −p 1 1 where It is easy to observe that if 1 in (53) is positive definite and if with defined in (42), then the functioṅ( , e,ė ) is negative semidefinite. Besides, notice that the matrix 1 in (53) can be rewritten as ] .
If and are positive definite then 1 is also positive definite. By applying Sylvester's criterion, the matrix is positive definite if while is positive definite if Therefore, the selection of large enough control gains Δ, and guarantee the existence of satisfying (43) so that the function ( , e,ė ) in (45) is positive definite and radially unbounded, anḋ( , e,ė ) in (48) is a negative semidefinite function. Hence the trajectories e( ),ė ( ),̃( ), and̃( ) of the overall closed-loop system (40)-(32) are uniformly bounded.
By integrating both sides of (51) it is possible to prove that where p 1 ( ) is defined in (52). Then, by invoking Barbalat's lemma [31], the limit (44) is assured.
The Scientific World Journal

Experimental Results
Two sets of experiments in different experimental set-ups have been carried out in order to assess the performance of the proposed adaptive neural network controller (28) and (32)- (33). The used experimental systems are (i) a horizontal two-degree-of-freedom arm, (ii) a vertical one-degree-of-freedom arm, which is affected by the gravitational force.
In each set of experiments the applied torque (28) is implemented with and without adaptive neural network compensation. The purpose of the experiments is to show the benefit of the adaptive neural network feedforward compensation.

Planar Two-Degree-of-Freedom Robot.
To carry out experiments, a planar two-degree-of-freedom direct-driven arm has been used. See Figure 1 for a CAD drawing and picture. Two experiments have been conducted corresponding to the implementation of a nonlinear PD controller which does not consider any adaptation and the proposed adaptive neural network-based controller in (28).
In reference to Tables 1 and 2, and the model (13), the elements of the experimental robot model are It is noteworthy that the robot is moving in the horizontal plane, whereby the vector of gravitational forces is null. Table 2 shows a numerical estimation of the parameters ∈ R, which was obtained by using the filtered dynamic model and the classical least-squares identification method; It is clear that the desired joint trajectory q ( ) satisfies the bounding assumption (22). See Figure 2 to observe the time evolution of the desired trajectory (62).

Results
. By removing the adaptive neural network compensation in the proposed controller (28), the following nonlinear PD-type controller is obtained: with r defined in (29). Hereafter, the controller (63) will be denoted as PD + Δ sign (r).
The PD + Δ sign (r) controller in (63) has been implemented in real time with the following control gains:   The results of implementing in real time the PD + Δ sign (r) controller in (63) can be appreciated in Figure 3, which shows the time evolution of the applied torques 1 ( ) and 2 ( ), and in Figure 4, that depicts the time evolution of the tracking error 1 ( ) and 2 ( ).
On the other hand, the new control scheme in (28) will be referred to as ANN + PD + Δ sign (r). This controller was implemented by using = 20, which is the number of neurons. The neural network requires the input vector The vector x ( ) describes the signals that are used in the inner layer of the neural network.
In order to establish a fair comparison scenery, the control gains (64) and (65), which were used in the PD + Δ sign (r) The Scientific World Journal  controller in (63), have been used in the real-time implementation of the proposed controller ANN + PD + Δ sign (r) in (28). Besides, the gains with meaning the identity matrix of dimension , were used the adaptation update laws (32) and (33), respectively.
The results of implementing the proposed controller ANN + PD + Δ sign (r) in (28) is shown in Figure 5, where the time evolution of the applied torques 1 ( ) and 2 ( ) are represented, and in Figure 6, where the obtained tracking errors 1 ( ) and 2 ( ) are described. Figures 4 and 6 show the tracking error e( ) ∈ R 2 for the controller PD + Δ sign (r) in (63), which does not consider adaptation, and the new scheme ANN + PD + Δ sign (r) in (28), which is based on the adaptation of neural network. From these figures the performance of the controllers can be assessed. For both implementations, one can consider a steady state behavior after 10 [sec]. In order to assess the performance of the controllers, the maximum peak-to-peak tracking error for each joint and for each implementation has been computed. Table 3 shows the performance of the two controllers, where the notation PTPV stands for "peak-to-peak value. " We can see that by using the new controller the tracking performance   Figure 7 shows the norm of the tracking error ‖ e( ) ‖, which is another form to compare the tracking performance of a controller with respect to the other. In this figure, the improvement in the tracking of the desired trajectory by using the ANN + PD + Δ sign (r) scheme in (28) is clearly observed. After 10 [sec], the peak value of the ‖ e( ) ‖ by using the PD + Δ sign (r) scheme in (63) is 0.161 [rad] and by using the ANN + PD + Δ sign (r) controller in (28) is 0.035 [rad]. Thus, the improvement using the new scheme is 78.3%.

Observations.
Finally, the output weightŝ1( ) ∈ R 10×1 and̂2( ) ∈ R 10×1 for joints 1 and 2, respectively, obtained in the real-time implementation of the new controller (28) are observed in Figure 8. A small value of such a weights is enough to improve the tracking performance drastically.

Vertical One Degree-of-Freedom
Robot. This experimental system consists in an Advanced Motion Controls servo amplifier operated in voltage mode and a Nema 34 brushed direct current motor which has attached a pendulum as shown in Figure 9. Like the two-degree-of-freedom robot, a data acquisition board and Matlab were used to implement the controllers in real time at 1 [kHz] sampling rate.
The model of this system corresponds to a pendulum with viscous friction which is actuated by a armature-controlled direct current motor, which can be written as [34] +̇+ sin ( ) = , where , , , , , and are strictly positive constants; denotes the armature current, and V is the voltage input.
The idea is to compare the performance and robustness of the controllers (69) and (70) in a system affected by the gravitational force and the electrical dynamics of the motor.
In this case, the model of the system of the robot was assumed to be unknown, whereby the tested controllers were tuned by trial and error until an acceptable performance was obtained. (71) Figure 9: Experimental one-degree-of-freedom arm (pendulum) subject to gravitational force. Notice that with trajectory the effect of the gravitational force is maximized. Figure 10 shows the time history of the desired joint trajectory ( ) in (71).

Results.
Like the case of the experiments with the twodegree-of-freedom robot, we have selected the control gains so that most of the control action is contributed by the adaptive neural network part of the control. Specifically, we selected the gains The adaptation of the parameters is given by using (32)  The proposed adaptive NN controller ANN + PD + sign ( ) was implemented by using = 10 neurons.
The results of the implementation of the controllers PD + sign ( ) in (69) and ANN + PD + sign ( ) in (70) are given in Figures 11 and 12, which show the time evolution of the applied voltage V( ) and the absolute value of tracking error | ( )|, respectively. The advantage of the proposed controller in ANN + PD + sign ( ) is clearly observed from includes the adaptive neural network, the absolute value of the tracking error | ( )| is reduced to values less than 0.08 [rad] after 20 [sec]. Finally, Figure 13 depicts the estimated input and output weights,̂( ) and̂( ), respectively, which remain bounded for all time.

Conclusions
This paper introduced a new adaptive neural network control algorithm for the tracking control of robot manipulators. The neural network of this controller uses only the desired joint position, velocity, and acceleration. The main theoretical result consisted in showing that the position and velocity error converge asymptotically while the input and output weights of the neural network remain bounded. The experimental tests showed the benefit of using an adaptive neural network plus nonlinear PD control action. The new controller showed robustness to model uncertainties and strong nonlinear effects such as gravitational effects and Coulomb friction.
Further research considers the application of the proposed ideas in PI joint velocity control [35], anticontrol of chaos [36], output feedback tracking control [37], and underactuated systems [38].