Application of Online Iterative Learning Tracking Control for Quadrotor UAVs Pong-in Pipatpaibul and

Quadrotor unmanned aerial vehicles (UAVs) have attracted considerable interest for various applications including search and rescue, environmentalmonitoring, and surveillance because of their agilities and small sizes.This paper proposes trajectory tracking control of UAVs utilizing online iterative learning control (ILC) methods that are known to be powerful for tasks performed repeatedly. PD online ILC and switching gain PD online ILC are used to perform a variety of manoeuvring such as take-off, smooth translation, and various circular trajectory motions in two and three dimensions. Simulation results prove the ability and effectiveness of the online ILCs to perform successfully certain missions in the presence of disturbances and uncertainties. It also demonstrates that the switching gain PD ILC ismuch effective than the PD online ILC in terms of fast convergence rates and smaller tracking errors.


Introduction
Unmanned aerial vehicles (UAVs) have become very popular among researchers and developers in the last decade, owing to their capabilities of various applications such as meteorological surveillance, disaster monitoring, and military purposes.Depending on their applications, UAVs vary in sizes, shapes, and operating ranges.UAVs are complicated for control considering the nonlinearity of the system, no supervision of pilots, and external disturbances that need sophisticated control system to deal with.
A quadrotor UAV is a special UAV that has four rotors with a symmetric shape to generate thrust.A quadrotor UAV can vertically take-off, hover, swiftly manoeuvre in any direction and carry a large payload comparing to its own weight .In addition, a quadrotor UAV is normally small compared with other types of UAVs, which makes it simple and cheap, and accessible indoors or in urban constrained areas.
In order to achieve autonomous control, many studies and experiments have been performed for UAVs.However, due to their nonlinearity, multi-input and coupling characteristics, traditional control methods such as PID control may perform poorly under uncertainty and wind disturbances for UAVs, as shown in [1][2][3].As an optimization technique with feedback control, Linear Quadratic Regulator (LQR) and its variation of State Dependent Riccati Equation (SDRE) control were proved to perform well for UAVs without disturbances [4,5].Comparisons and implementation of Sliding Mode Control (SMC) and Backstepping control were presented in [6][7][8].The SMC methods were found to be robust against uncertainties, but they are still relatively complicated and it might cause chattering problems.An H-infinity control [9] was proposed to deal with the problem of stabilization of a rotorcraft with small external disturbances.An adaptivefuzzy control was developed in [10] with robustness but limited to stabilization applications only.Neural Network control methods were implemented in [11,12] that proved to perform well, but intensive computations are required for training NN that may limit their real applications.A testbed for the development and demonstration of control systems and trajectory planning optimization of UAVs was set up and discussed in MIT [13,14].
On the other hand, iterative learning control (ILC) [16] is based on simple PD/PID control and adopts the idea of human learning process, and it is mainly used in robotic manipulators where their tasks are performed repeatedly.ILC Figure 1: An example of a quadrotor UAV [15].can improve tracking performances when a specific task is done again and again, and it is robust against uncertainties and disturbances.ILC was originally proposed by Arimoto [16], where only derivative part (D-type) and information from previous iteration(s) were used to determine the control input for current iteration (offline ILC).Some variations of ILC such as P-type [17], PD-type [18,19], PI-type [20,21], and PID-type [22,23] were studied, but most of them cannot assure monotonic convergence for nonlinear systems.Faster convergence rates can be achieved by using current iteration errors (online ILC) in [24][25][26][27] or a combination of both online and offline ILCs in [28][29][30][31].An even faster monotonic convergence rate and less tracking error can be achieved by using switching gain learning control developed in [32,33].Performance comparison of various types of ILCs was conducted in [34].
Many control methods have been developed to deal with the control problems of UAVs.The majority of those controllers focused heavily on the stabilization problem which is the first step toward successfully autonomous flights.Some of them also handled position maintaining or velocity holding in order to fulfill certain manoeuvrings and thus full autonomous control.Some researches focused on the trajectory tracking control of UAVs through developing different control methods [35][36][37].Based on gain-scheduling control theory, the nonlinear problem of UAV dynamics was represented as piece-wise error dynamics over a predefined set of operating regions in [35], and the design and performance evaluation of a trajectory tracking controller were discussed.LQR control was applied as a trajectory follower to minimize errors between the real trajectory and the reference trajectory in [36].A switching control method was introduced for trajectory tracking of fixed-wing UAVs in [37].
To the best of our knowledge, there is rare application of ILC on the control of UAVs.The contribution of this work to the UAV field consists in the exploration of different online ILCs on trajectory tracking control of quadrotor UAVs for the first time, focusing on performance improvements under conditions of uncertainty and disturbances.The capability of online ILCs to reject random disturbances and noise is examined and demonstrated.The research can be viewed as a new application of online ILCs.In this research, two competitive types of ILCs, namely, PD online ILC and switching gain PD (SPD online ILC), are employed in order to control a quadrotor UAV to follow various desired trajectories in the presence of disturbances and uncertainties.Nonlinear dynamic model of the quadrotor UAV is established where major aerodynamic and gyroscopic effects are included.Simulations are conducted to verify the capability of performing various missions repeatedly with some constraints.

Quadrotor UAV: Dynamic Model and Parameters
As named, a quadrotor UAV consists of four rotors producing thrust upward against its own weight and payloads.Usually, payloads are placed at the center of the UAV, and rotors propelled by DC motors with or without gearbox.Figure 1 shows an example of a quadrotor UAV and Figure 2 illustrates the coordinate system and positions of rotors used in this paper.The rotors are divided into two pairs; one pair rotates in the opposite direction of the other pair.Increasing the thrust on one side and decreasing the thrust on the other side of the same pair will result in rotation in the pitch or roll direction and the quadrotor will tend to translate toward the direction that it inclines to.Simply increasing thrust equally in one pair and decreasing thrust in the other pair will result in the rotation of the yaw angle while maintaining the position and altitude.
According to the coordinate system shown in Figure 2, a nonlinear translational and rotational dynamic model for the quadrotor UAV can be expressed as [12] =J ω+ (J+J  Ω  e 3 ) +d  , where  is a torque vector of the three axes; J is the inertia tensor of a quadrotor;  is an angular velocity vector; J  is the moment of inertia of rotor; is the total rotor speed; e 3 = [0 0 1]  is a unit vector in the inertial frame; R is the rotation matrix; T is a force vector; d  and d  are the disturbance vectors for rotations and translations, respectively.The equations of motion, along with control inputs, can be described in each axis based on (1) as [12] where  is the span of the quadrotor;  is the thrust coefficient;  is the drag coefficient, which relates to torque in the yaw angle.
The control inputs related to each rotor speed are defined as follows: Note that (2)-(4) are derived using the rotation matrix of   instead of   as used by most robotic applications.Design parameters and some limitations of a quadrotor UAV are referred from a real application in [15] and listed in Table 1.
To design a control system for the quadrotor UAV, it is more convenient to establish a state-space model.First, we define a state variable vector as follows: Then the state space system for the UAV dynamic model described in (2)- (7) ) ) ) , where In this paper, the disturbances given in (10) have wind disturbance components along the  and  directions that are modeled by a correlated Gauss-Markov process [9].In all other directions, some random noises are added to simulate the disturbances and the uncertainties of the dynamics.

Different ILCS: Basics and Convergence Conditions
ILC can improve tracking performance of a nonlinear system through iterative operations when a specific task is performed repeatedly.Consider a nonlinear time varying system with the following form: where subscription  denotes the iteration number.
A general form of the control update law for ILCs involving online and offline learning technique can be written    as where () =   ()−() and ė () = ẏ  ()− ẏ ().  ,   , and   are PID control gains, and subscriptions on and off represent online (feedback, using errors from the current iteration) and offline (feed-forward, using errors from the previous iteration), respectively.Based on the selection of the control  gains, ILCs can be mainly classified as three types, namely, offline, online, and online-offline.In the following sections, the convergence conditions and the convergence rate are discussed.

Offline ILCs.
Traditional ILCs belong to offline learning control, where only the errors from previous iterations have been included in the control law: In the case that only a proportional part is utilized (P-ILCs), the control law can be expressed as For a P-type ILC, the convergence condition for the controlled system ( 12) is [17] Whenever a derivative part is involved in an ILC, including D-ILC, PD-ILC, and PID-ILC, the convergence condition is [16,18,19] The two aforementioned convergence conditions imply that offline learning gains are upper bounded, which means that only a certain range of control gains can be chosen.Therefore, offline ILCs have a relatively slow convergence rate.

Online ILCs. On the other hand, online ILCs only rely on current iteration errors:
and their convergence conditions are [27][28][29]   on =       ( +   on ) −1      < 1 (P-type only) , It is noticed that the learning gains are unbounded for online ILCs from (19), which makes it very flexible to choose control gains.Furthermore, it is demonstrated that the larger the online control gains, the faster the system convergence rate.

Online-Offline ILCs.
When online and offline ILCs are combined and used simultaneously, it simply turns into online-offline ILCs.Convergence conditions are then a combination of both types and can be expressed as [31]   off < 1   on < 1 (P-type only) or

Controller Design for Quadrotor UAV
In this paper, the integral part of PID control in (13) will be neglected as ILC itself has the feature of integration, by adding control actions from previous iterations to the current one.Furthermore, offline and online-offline ILCs are not considered as many researchers have demonstrated a superior performance of online ILCs over offline ILCs. Figure 3 shows a scheme of a control diagram for online ILCs.Only ordinary PD online ILCs and SPD online ILCs will be tested and compared in this research.
According to (8), there are four control inputs for the UAV system.Based on the state variables defined in (9), the control variables are defined as  3 to  6 , corresponding to  axis and three rotation angles; their corresponding derivatives are  9 to  12 .The control inputs of the UAV are calculated using the following ILC update law: where (  on )  and (  on )  are the corresponding online learning proportional and derivative gains.
If the  direction needs to be controlled, from (22), we can see that  3, and  9, are the only desired states determined by the desired trajectories.In order to find properly desired states as a function of time for the rest of state variables, two dummy control inputs for the  and  direction motions are defined as where  1, ,  2, ,  7, , and  8, are the desired trajectories in the  and  directions and their derivatives, respectively; (  on )  and (  on )  are the online learning gains.Based on the defined rotation matrix R, the desired angles can be calculated from Since a quadrotor UAV is very agile and is able to maneuver in any direction, the yaw angle in this paper is defined as According to (19), the convergence conditions for online ILCs are only related to matrices B (input matrix) and C (output matrix) and are not related to function (, ) in (12).This feature provides the robustness of the control law to deal with the mismatch in modeling the system.Also, the choices of control gains in ( 22)-( 23) are flexible because they are unbounded as long as (19) is satisfied.It should be noticed that, in the developed control law ( 22)-( 23), the angular positions and angular speeds should be measured by sensors.A fibre optic gyroscope sensor can be used for this purpose because of its high accuracy, fast response, and low weight.
Note that in the SPD ILC, the online learning gains are defined as functions of iteration number and are chosen as The switching action here occurs in iteration domain rather than in time domain.For traditional switching control, the switching action may cause problems in transient process of the switched system, while this phenomenon does not occur in iteration domain.

Simulation Results
In order to obtain a smooth trajectory, that is, continuous in position, velocity, and acceleration, a 5th order polynomial ISRN Robotics In the simulation studies, two different types of disturbances are introduced to deal with different environmental effects on different direction motions of the quadrotor UAV.
For the  and  direction motions, external disturbances caused by wind gusts are described by stochastic functions where a Gauss-Markov process is chosen to simulate the wind effect to the UAV system   (  ) =   ( −1 ) +  * rand () ,   (  ) =   ( −1 ) +  * rand () .Throughout all the simulations, disturbances in the  direction and all three angles are presented in (29).Note that the disturbances with random noise are very large compared to the components of the dynamic model  = 10 + 20 sin (2) + rand () ,   = 0.1, To demonstrate tracking performance and robustness of online ILCs, the following simulation will cover main missions of a quadrotor UAV, that is, take-off, hovering, and moving in various trajectories.

Take-Off Control.
In a take-off mission, the quadrotor UAV is commanded to smoothly take-off and ultimately hover at a desired height ℎ  = 10 m within a time duration of  = 5 sec.The desired trajectory is defined as and a set of online learning gains is chosen as where  on = 0.5, 1, 1.5 for different control gains.For most simulations conducted in this research, we set  on = 1 unless otherwise stated.The choice of control gains above was based on trial and error, with regard to stability and trajectory tracking performance.
The constant  on is used to demonstrate the effect of increasing-decreasing the whole gain set by a factor.A variation of learning gains and its trajectory tracking performance for take-off are listed in Table 2.It should be mentioned that, in order to get fair comparison results, the disturbances and uncertainties for each simulation in the same iteration are assumed to be the same.
As shown in Table 2, reducing (  on ) 1 and (  on ) 2 by a factor of 10 significantly reduces the convergence rate and yields much larger errors in the first three iterations.Likewise, decreasing (  on ) 4 and (  on ) 5 by a factor of 10 yields similar results.This is due to a slower reaction in attitude control, making the quadrotor UAV away from the desired position in the presence of disturbances.Lastly, reducing (  on ) 6 by a factor of 10 shows a considerable decrease in yaw angle stability and a slower convergence rate.The effect of changes in   on on the performance is illustrated in Figure 4.For the  and  axes tracking, when   on =   on the system yields larger errors and tends to produce steady errors.For   on = 1.5  on and   on = 3  on , the results are much better in terms of maximum errors and final steady errors.These results strongly show the importance of the derivative gains.For the  direction, the results are merely identical.Figure 5 shows how changing the whole set of gains by a factor affects trajectory tracking performances.With  on = 0.5, for the  and  directions, it is obvious that the control gains are not high enough to ensure monotonic convergence and yield large errors.On the other hand, when  on = 1 and  on = 1.5, the system is monotonically convergent and even faster for the latter case.Again, the effect is hardly detected in the  direction.This is because control in the  direction depends solely on  1 , while the  and  motions also depend on the attitude of the quadrotor UAV.
To compare the performances of ILCs with traditional control such as LQR control, take-off simulations were fulfilled.Table 3 lists the maximum tracking errors under different LQR performance indices where an approach for the design of LQR controller discussed in [4] is used.Comparing the maximum errors in  direction shown in Table 3 with those listed in Table 2, it is clearly shown that the ILC performed much better than the LQR.Such a result would be expected as the ILCs is a kind of nonlinear learning control while the LQR is a linearized control for a nonlinear system.After finishing the initial comparison study on the performance improvements by changing online control gains, some take-off simulation studies with wind disturbances and uncertainties are conducted.Figure 6 shows the maximum tracking errors from iteration to iteration controlled by both PD online ILCs and SPD ILCs.In the presence of disturbances, there are small errors for the  and  direction motions controlled by the PD online ILC because of the existence of wind disturbance.But SPD ILCs outperform PD online ILCs with their incredibly fast convergence rates, and achieve very good performances in all three translation motions after 3 iterations.It demonstrates the capability of dealing with disturbances in SPD ILC.
Trajectory tracking performance in translations and orientations controlled by PD online ILCs is shown in Figure 7 for different iterations.It can be seen that the quadrotor UAV can smoothly take-off, with some errors in the  and  directions that were caused by the wind disturbances.It is also noticed that the PD online ILCs and SPD ILCs can reject disturbances in orientations.Figure 8 shows the tracking performance improvements in  direction from iteration to iteration for both online ILCs.It clearly shows the significant differences between the PD online ILC and the SPD ILC after the first iteration.Rotor speed in the 9th iteration is depicted in Figure 9 for the PD online ILC, which shows that all four rotors operate in the specified ranges.

Smooth Translation
Two sets of simulations are performed here.The first simulation is set for a short range linear translation motion with  = 10 m and  = 5 sec., and the second one is for a long range translation motion at a higher speed with  = 100 m and  = 15 sec.
For the first set motion, it shows that the PD online ILCs tend to take more iterations, as many as 9, to converge to approximate zero errors in  and  directions while SPD ILCs take 6 iterations.Trajectory tracking errors from iteration to iteration are shown in Figure 10.It clearly shows that the SPD ILC can ensure less tracking errors and faster convergence rates.
Figures 11 and 12 show the tracking performance improvements from iteration to iteration for both PD online ILC and SPD ILC.The effectiveness of the online ILCs in terms of tracking error reductions and resistance to disturbances is clearly shown.
Note that shown in Figure 13, there are relatively large errors in the  and  directions for the first iteration and then they converge to stable error boundaries for the PD ILC, but the SPD ILCs obtained very good tracking performance with very small (<10 −2 m) tracking errors in all three directions even under the wind gust disturbance conditions.
For the second set test with large motion ranges and fast speeds, the maximum tracking errors from iteration to iteration are depicted in Figure 14 for both PD online ILC and SPD ILC.It is shown that the PD online ILCs obtain acceptable tracking performance (the tracking errors are around 1 m for both  and  directions), while the SPD ILCs still perform much better than the PD online ILCs.
Figure 15 shows the tracking errors in the  and  directions from iteration to iteration for both PD online ILC and SPD online ILC.Considering the long range motions and relatively higher speed than the previous short range motion case, it is clearly shown that, after a few iterations, the quadrotor can obtain smooth and good motions that follow the desired trajectories, especially for the SPD ILC.
Furthermore, to examine the tracking performance for a three dimensional translation motion based on the ILCs, a simulation of linear motion from point A (0, 0, 0) to point B (100, 100, 10) in 15 seconds is conducted.Figure 16 shows the maximum tracking errors from iteration to iteration controlled by these two different online ILCs.It is clearly shown that the SPD ILCs performed much better than the PD online ILCs in terms of fast convergent rate, small tracking errors, and good capability to compensate the disturbances and uncertainties.It also shows that the tracking performance in the  direction was controlled very well for both online ILCs. Figure 17 shows the tracking errors from iteration to iteration for both PD online ILC and SPD online ILC for 3D translation motions.It demonstrates the effectiveness of the online ILCs and the better tacking performance obtained by the SPD online ILC.

Circular Trajectories Tracking.
To prove that the quadrotor UAV can freely maneuver a complex trajectory, a circular trajectory tracking test is performed by using the online ILCs.In this mission, the quadrotor revolves the - plane with a radius  = 20 m for a time duration  = 20 sec., smoothly starting from and ending at the velocity of 0.
Figure 18 shows the performance improvements from iteration to iteration for both PD online ILCs and SPD ILCs.Once again, it demonstrated the effectiveness of the SPD ILCs. Figure 19 shows the tracking errors controlled by PD online ILCs and SPD ILCs for the - plane circular trajectory, respectively.It can be observed that PD online ILCs can obtain acceptable tracking performances through iterative learning processes, and SPD ILCs converge to approximate zero errors at the 3rd iteration while PD online ILCs still have small bounded tracking errors.Therefore, it can be seen that SPD ILCs perform much better in yaw angle than the PD online ILCs.
By adding the  direction motion (20 m) accordingly with the - plane circular motion discussed above, a 3D circular tracking case was formed and used to examine the effectiveness of the online ILCs. Figure 20 shows the maximum tracking errors in three axes for the PD online ILCs and the SPD ILCs, while Figure 21 shows the tracking performance improvements from iteration to iteration for both online ILCs.It demonstrates again that the SPD ILCs can obtain much better tracking performances than the PD online ILCs under the existence of disturbances and uncertainties.

Conclusion and Discussion
Iterative learning control is simple to build, easy to implement, and robust to uncertainties and disturbances.In order to perform some fundamental missions for a quadrotor UAV, some trajectory tracking control for a UAV was conducted based on online ILCs in this paper.Different trajectory tracking tasks using PD online ILCs and SPD ILCs were simulated and compared.It was proven that tracking performances can be improved by choosing large control gains for the online ILCs.In addition, the choice of control gains is very flexible.Despite nonlinearity of the system and the presence of disturbances, both types of online ILCs demonstrated good tracking capabilities of smooth take-off, straight-line, and circular motions in two and three dimensions and provided good performances after a few iterations.Simulation results demonstrated that SPD ILCs performed much better than PD online ILCs in terms of convergence rate and disturbance rejection.
The online ILCs have been successfully applied to trajectory tracking control of a quadrotor UAV through simulations for different missions.To deal with the real application of online ILCs, it should be mentioned that a PD feedback control is applied to control the quadrotor UAV in the first iteration, as there is no previous experimental learning process.The PD feedback control was proved to provide the stability of the UAV system, even with some big tracking errors.The control input information obtained from the first iteration will be stored and used as a feedforward control for the next iterations.In case the state of the system cannot be measured directly, the online ILCs update law could include a Kalman filter to estimate the states.With all the aspects shown above, implementation to a real application seems to be practical, and it is a future work.

Figure 3 :
Figure 3: Control scheme for PD online ILC and SPD ILC.

Figure 4 :
Figure 4: Effect of   on tracking errors for PD online ILCs.

Figure 5 :
Figure 5: Effect of changing the whole set of control gains by a factor of 0.5 and 1.5.

Figure 6 :
Figure 6: Tracking performance comparison between PD online ILC and SPD ILC for take-off.

Figure 16 :
Figure 16: Trajectory tracking performance comparison between PD online ILC and SPD ILC for -- translation.

Figure 17 :
Figure 17: Trajectory tracking performance comparison between PD online ILS and SPD ILC for -- translation.

Figure 18 :
Figure 18: Trajectory tracking performance comparisons for - plane circular motion.

Table 1 :
Parameters of a quadrotor UAV.

Table 2 :
Effect of each control gain on tracking errors.

Table 3 :
Maximum tracking errors in the  direction for take-off under LQR control.