Higher-Order Iterative Learning Control with Optimal Control Gains Based on Evolutionary Algorithm for Nonlinear System

For the nonlinear discrete-time system, higher-order iterative learning control (HOILC) with optimal control gains based on evolutionary algorithm (EA) is developed in this paper. Since the updating actions are constituted by the tracking information from several previous iterations, the suitably designed HOILC schemes with appropriate control gains usually achieve fast convergence speed. To optimize the control gains in HOILC approach, EA is introduced. *e encoding strategy, population initialization, and fitness function in EA are designed according to the HOILC characteristics. With the global optimization of EA, the optimal control gains of HOILC are selected adaptively so that the number of convergence iteration is reduced in ILC process. It is shown in simulation that the sum absolute error, total square error, and maximum absolute error of tracking in the proposed HOILC based on EA are convergent faster than those in conventional HOILC.


Introduction
In real applications such as robot manipulator systems [1][2][3][4][5] and flexible systems [6][7][8], there are many unmanned autonomous systems in complex environments. e exact mathematical model is hard to construct. For these systems, iterative learning control (ILC) is proposed. It is an effective intelligent control approach applied in dynamical systems that perform repetitive tasks to track a specific trajectory in a certain time interval. By using the control input and tracking information of previous iterations, the control input signal can be gradually updated from iteration to iteration such that the tracking performance can be improved. Less previous knowledge about the controlled systems makes ILC popular in theoretical fields [9][10][11][12][13][14] as well as applicable fields [15][16][17][18][19].
First-order ILC, which generates the control input from tracking information at last iteration, is widely applied to dynamical systems for perfect tracking in a finite time interval [20][21][22][23][24][25][26]. However, only the tracking information of last iteration is utilized to update the current control input in first-order ILC, and thus it is difficult to obtain a satisfactory convergence speed. To achieve faster convergence speed, higher-order ILC (HOILC) adopting the tracking information of many previous iterations to generate the current control input signal was proposed [27][28][29][30][31]. Since the updating actions are constituted by the tracking information from several previous iterations, the tracking performance of suitably designed HOILC is better than that of first-order ILC. Specifically, the appropriate control gains can accelerate the convergence process of HOILC.
us, how to select optimal control gains is a significant issue in HOILC designs.
Motivated by the above observation, in this paper, the evolutionary algorithm (EA) originating from biological evolutionism is adopted to choose the optimal control gains in HOILC scheme adaptively. EA is a heuristic optimizing algorithm which simulates the reproduction, selection, crossover, and mutation in biological evolution process. It has been widely introduced to deal with various optimal issues [32][33][34]. In this paper, the encoding strategy, population initialization, and fitness function of EA are designed according to the HOILC characteristics such that the generations in EA are reduced. en, the designed EA is involved in HOILC to optimize the control gains. After that, the optimal control gains and the control inputs are generated simultaneously. Comparing with traditional HOILC, the number of convergence iteration is reduced in the proposed EA scheme based HOILC (EA-HOILC). e EA with global optimization is introduced to optimize the control gains of HOILC in this paper. e rest of paper is organized as follows. e problem formulation is given in Section 2.
e HOILC with its convergence analysis is provided in Section 3. Section 4 presents the designed EA-HOILC scheme with optimal control gains. In Section 5, an example is provided to illustrate the effectiveness of the proposed EA-HOILC. Section 6 concludes this paper.

Problem Formulation
Consider the following nonlinear discrete-time system which performs repetitive operation: where j ∈ 0, 1, 2, . . . { } and i ∈ 0, 1, . . . , M { } represent the iteration index and the time point, respectively. x j (i) ∈ R w , u j (i) ∈ R, and y j (i) ∈ R denote the state, control input, and output of system (1), respectively.
is the ILC tracking error at j th iteration for i ∈ 0, 1, . . . , M { }. e following assumptions are required for the technical analysis. ‖ · ‖ represents the required norm in this paper.
Assumption 1. For all j, the initial state satisfies As the identical initial condition considered in Assumption 1 cannot be satisfied, the techniques proposed in [29][30][31] can be introduced to deal with the vibration of initial state.

Assumption 2.
e nonlinear function f(·, ·) in system (1) is assumed to be differentiable to i and to be globally Lipschitz in the first variable, that is, ∀x where c f > 0 is the Lipschitz constant. Remark 1. It is noted that Assumption 3 implies that the relative degree of system (1) is one. For the nonlinear discrete-time systems with higher relative degree, the ILC law can be modified according to the order of system relative degree as discussed in [31]. Suppose that the reference output y d (i) is realizable, there exists a unique control input u d (i) ∈ R such that (4) e objective of this paper is to develop an EA-HOILC method, which generates the control input from the tracking information of several previous iterations. e control gains are optimized by EA to reduce the number of convergence iteration. For HOILC convergence analysis, the following lemma is adopted.
Lemma 1 (see [31]). Let g j be a real sequence defined as then lim j⟶+∞ d j � 0 implies that lim j⟶+∞ g j � 0.

HOILC Design and Convergence Analysis
In this section, for nonlinear discrete-time system (1) under Assumptions 1-3, the following HOILC law is designed for where N ≥ 1 is the order of HOILC law (7), and W v ∈ R and . . , N are the control gains.

Remark 2.
In the existing HOILC schemes [30,31], the initial control inputs u 0 (i), u 1 (i), . . . , u N−1 (i) are normally set as zero vectors. In this paper, since the control inputs can be obtained by EA along with the optimal control gains, we can set the initial control inputs u 0 (i), u 1 (i), . . . , u N−1 (i) same as the generated control inputs. It means that the initial control inputs are optimized by EA, which can also accelerate the convergence speed. (7) is applied. If the control

EA-HOILC Scheme with Optimal Control Gains
eorem 1 provides the asymptotic convergence of the proposed HOILC. It is well known that the control gains can affect the convergence performance significantly. In this section, the control gains of the HOILC developed in Section 3 are optimized by EA to reduce the number of convergence iteration.
EA is an intelligent optimization algorithm which simulates the process of biological evolution to gain the optimal solution. e main idea of EA-HOILC is presented as follows.

Encoding Strategy.
In this paper, the control gains of HOILC are real numbers, so it is appropriate to choose the real encoding strategy. e control gains to be optimized in HOILC law (7) are W v and L v (v � 1, 2, . . . , N). Since the convergence condition (8) holds, it is easily obtained that As a result, we can assume the variable vector in EA to be λ ∈ R 1×(2N−1) , and the encoding strategy is represented as

Population Initialization and Individual Evaluation.
Based on the convergence conditions (8)-(10), the value range of control gains W v and L v for v � 1, 2, . . . , N could be determined. us, the initial population can be produced according to the convergent conditions. Let P be the population size, without loss of generality, assume that P is even. e variable vector of h th individual in the population is represented as λ h � W 1,h W 2,h · · · W N−1,h L 1,h L 2,h · · · L N,h ] ∈ R 1×(2N−1) which are initialized to λ ini h ∈ R 1×(2N−1) for h � 1, 2, . . . , P. On the other hand, for variable λ h , let the system output for h th individual at i th time point be Y h (i, λ h ). To evaluate the individual superiority, the following fitness function fit h (λ h ) ∈ R of h th individual is established: where M is a constant large enough and E h (λ h ) is the sum of absolute value of tracking error represented as From the fitness function (27) and the initial variable λ ini h , the initial fitness value of h th individual fit ini h is obtained. en, we have the following initial fitness vector fit ini ∈ R P of population: Hence, the initial population pop ini ∈ R P×2N is constructed as where the initial variable of population is From (31), the α th (1 ≤ α ≤ 2N − 1) variable of the β th (1 ≤ β ≤ P) individual is represented as the β th row of α th column. e last column of pop ini shown in (30) is the fitness value of the corresponding initial variable in population.

Selection Strategy.
e individuals are selected into next generation by roulette strategy and elitism strategy. e individual with bigger fitness value is selected at higher probability by the roulette strategy. However, one shortcoming of the roulette strategy is that the best individual in old population might be missed. So, we adopt elitism strategy to ensure that the best individual of last generation can be retained. Due to these two strategies, the number of convergence generation of EA can be reduced.

Crossover Operator.
e crossover probability 0 < p c < 1 depends whether an individual needs to cross. For h ′ th individual, where h ′ � 1, 3, 5, . . . , P − 1, a random number is produced between 0 and 1, which is represented as r h′ . If r h′ > p c , the crossover operation occurs. Otherwise, the crossover operation does not occur. Due to the real encoding strategy, arithmetical crossover operator is adopted. Assume the variable vectors of h ′ th and h ′ + 1 th parent individuals to be par λ h′ and par λ h′+1 , respectively, which are selected to cross. After crossover, they generate two new individuals, of which the variable vectors are represented as chi λ h′ and chi λ h′+1 . For h ′ � 1, 3, 5, . . . , P − 1, the crossover operation is expressed as where η h′ ∈ (0, 1) is the crossover weighting for the h ′ th and h ′ + 1 th parent individuals.

Mutation Operator.
In this paper, we adopt the stochastic mutation strategy. Let 0 < p m < 1 be the mutation probability. For h th individual, a number is produced randomly between 0 and 1 represented as r h , h � 1, 2, . . . , P.
If r h > p m , the mutation operation occurs. Otherwise, the mutation operation does not occur. Let par λ h be the variable vector of h th individual which is selected to mutate. After mutation, a new variable vector chi λ h is produced. e mutation operator is defined as where m h ∈ R 1×(2N−1) is the mutation weighting for h th individual.

Terminative
Conditions. e terminative conditions can be determined by the fitness value or by the tracking error. In simulation, the number of generation in EA is set to be 100. Finally, we can obtain the optimal control gains W best v (v � 1, 2, . . . , N − 1) and L best v (v � 1, 2, . . . , N) from the best individual produced by EA. According to the convergence condition (8), the last control gain W best N is derived by

Overview of the Proposed EA-HOILC.
e flowchart of the proposed EA-HOILC is depicted in Figure 1. First of all, according to the control gains characteristics and convergence conditions (9) and (10), the initial variable λ ini h (h � 1, 2, . . . , P) is obtained. en, we apply the traditional HOILC at i ∈ 0, 1, . . . , M { } with initial control gains in each λ ini h , where convergence condition (8) is considered. By using the tracking error E h (λ ini h ) with λ ini h as shown in (29) produced by HOILC, the corresponding fitness fit ini h (h � 1, 2, . . . , P) is derived from (27). Combining λ ini h and fit ini h for h � 1, 2, . . . , P, the initial population pop ini as shown in (31) with (30) and (32) is produced. Secondly, selection, crossover, and mutation are performed by the selection strategy, crossover operator, and mutation operator, respectively. After that, a new input u p (i), i ∈ 0, 1, · · · , M { }, is obtained along with the optimal control gains. Set the initial control inputs of the EA-HOILC u 0 (i) � u 1 (i) � · · · � u N−1 (i) � u p (i), i � 0, 1, . . . , M. en, the HOILC with optimal control gains process begins.

Simulation
To verify the effectiveness of the proposed EA-HOILC, a two-link robotic fish is employed. e system dynamic of the two-link robotic fish is described as follows [22]: where M f � 0.4 kg is the mass of robotic fish, ξ � 165.7056 kg/m is the water resistance coefficient, V j (t) denotes the velocity, and F j (t) is the forward thrust produced by the tail motion. Let the velocity V j (i · T S ) and the forward thrust F j (i · T S ) be the system state x j (i) and the control input u j (i), respectively, where T S � 0.1 is the sampling time. We can discretize system (34) by using _ V j (i · T S ) ≈ (V j ((i + 1) · T S ) − V j (i · T S ))/T S . us, the discrete-time system is where A f � −ξ · T S /M f and B f � T S /M f . e reference output trajectory is represented as with M � 99. For the proposed HOILC algorithm, set the order N � 2. e control gains W 1 , L 1 , and L 2 are selected by EA. Another control gain W 2 � 1 − W 1 is obtained from convergence condition (8). e crossover probability p c � 0.7, and the mutation probability p m � 0.3. To evaluate the tracking performance, three tracking indexes on sum absolute error SE k , total square error TE k , and maximum absolute error ME k are defined as follows: In the simulation, the EA-HOILC is run 10 times, and the optimized control gains are shown in Table 1. Figure 2 exhibits the system output performance at iterations j � 15, j � 22, and j � 39 by using EA-HOILC with the average values of optimal control gains in 10 times. To compare the convergence speed between EA-HOILC and conventional HOILC with different parameters, the control gains in HOILC proposed in [30] with 2-order are, respectively, chosen as following two cases. Case 1: W 1 � 0.7, W 2 � 0.3, L 1 � 0.3, and L 2 � 0.2 and Case 2: W 1 � 0.6, W 2 � 0.4, L 1 � 0.2, and L 2 � 0.1. e corresponding sum absolute error SE k , total square error TE k , and maximum absolute error ME k of tracking are shown in Figure 3. From Figure 3, one can observe that the case with lager control gains in W 1 , L 1 , and L 2 can achieve faster convergent speed in conventional HOILC. Moreover, it is clearly revealed that the proposed EA-HOILC can make the convergence iterations less than the conventional HOILC with the same order.

Conclusions
In this paper, an HOILC law utilizing the tracking information of several previous iterations is proposed for the nonlinear discrete-time system. e convergence is rigorously analyzed based on the mathematical induction. In order to improve the convergence performance of the developed HOILC, the EA with global optimization is introduced to optimize the control gains. With the optimal control gains, the proposed EA-HOILC can achieve faster convergence speed. In simulation, it is shown that the sum absolute error, total square error, and maximum absolute error of tracking in EA-HOILC are convergent faster than those in the conventional HOILC with same order. However, it is worth noting that because EA is adopted to select the control gains, the offline computing time of EA-HOILC is longer. It is very suitable for the cases in which fewer iterations are required only. For instance, to destroy a target with bombs, the proposed method can reduce the number of bombs at the cost of computing time. Future research will extend the EA-HOILC developed in this paper to the dynamical systems with uncertainties in real application [35][36][37].

Data Availability
e data used to support the findings of this study are included within the article.

Conflicts of Interest
e authors declare that there are no conflicts of interest.