Multiple Model ILC for Continuous-Time Nonlinear Systems

and Applied Analysis 3 (1) 󵄩󵄩󵄩 󵄩 󵄩 I − ̃ B(x(t), t)B(x(t), t)󵄩󵄩 󵄩 󵄩 ⩽ ρ < 1 (x, t ∈ R × [0, T]), (2) xk(0) = xd(0) (k = 0, 1, 2, . . .), then, for arbitrarily given and achievable desired trajectory xd(t) (t ∈ [0, T]), ILC learning law (5) can generate a control sequence which guarantees xk(t) to converge to xd(t); that is, lim k→∞ 󵄩 󵄩 󵄩 󵄩 xd (t) − xk (t) 󵄩 󵄩 󵄩 󵄩 = 0 (t ∈ [0, T]) . (6) The proof of Lemma 1 is attached at the end of this paper as the Appendix. Further, from the proof, we can draw the following conclusion. Corollary 2 (see [24]). Denote Δxk(t) = xd(t) − xk(t), Δuk(t) = ud(t) − uk(t), L(x, t) = ̃ B(x, t), and h(x, t) = L(x, t)[ẋd(t) −̃f(x, t)]. As ̃ B and ̃ B̃f are Lipschitz continuous about x; h satisfies the Lipschitz condition; that is, for all t ∈ [0, T], and x1, x2 ∈ R, 󵄩 󵄩 󵄩 󵄩 h (x1, t) − h (x2, t) 󵄩 󵄩 󵄩 󵄩 ⩽ kh 󵄩 󵄩 󵄩 󵄩 x1 − x2 󵄩 󵄩 󵄩 󵄩 , (7) where kh is the Lipschitz coefficient of h. For single model ILC, the proof of Lemma 1 gives that 󵄩 󵄩 󵄩 󵄩 Δxk 󵄩 󵄩 󵄩 󵄩 ⩽ bB ∫ t 0 e b 2 (t−τ) 󵄩 󵄩 󵄩 󵄩 Δuk 󵄩 󵄩 󵄩 󵄩 dτ, (8a) 󵄩 󵄩 󵄩 󵄩 Δuk+1 󵄩 󵄩 󵄩 󵄩λ ⩽ (ρ + b1bBλb 2 ) 󵄩 󵄩 󵄩 󵄩 Δuk 󵄩 󵄩 󵄩 󵄩λ , (8b) where b1 = kh + bL(kf + kBbud), b2 = kf + kBbud, λb 2 =


Introduction
Data-driven techniques, which can extract and utilize useful information from huge amount of available process data to realize the control of complex system, have been one of the hottest research focuses [1,2].As one of the most representative data-driven methods, iterative learning control (ILC) can use the information of measured state error and control input to realize desired control performance through iterative refinement of control signal [3].Just relaying on the process measurements without prior knowledge of the controlled plant, ILC is an effective control strategy to realize the uniform tracking of desired trajectory, even for systems with uncertainties like imprecise parameters and unmodeled dynamics.
However, it should be noted that, for traditional ILC methods since Arimoto et al. [4], strict iteration-invariant identical conditions on initial states and process dynamics, as well as the desired trajectory, must be satisfied [5].ILC has been applied to robot manipulator [6], chemical process [7], and semiconductor manufacturing [8] based on these identical conditions.However, in practical control process, the strict conditions are always difficult to be guaranteed and hinder the further applications of ILC.
Recently, how to improve ILC to deal with iterationvarying factors has been an important research focus [9].
As we know, iterative learning control is a 2-dimensional problem working on the time axis and iteration axis; model uncertainty in the iteration domain will increase the difficulty in the 2D scheme, while traditional ILC is based on the iteration-invariant premise.Meanwhile, as the system dynamic will be changed due to iteration-variance in parameters, it is a relatively difficult problem compared with other kinds of nonrepetitiveness [10][11][12].
The first representative approach dealing with iterationvarying parameters in ILC is the norm-based method.For global Lipschitz continuous system, the 2D problem can be formulated into a contraction mapping problem, and the convergence is analyzed by the norm-based theory.Paper [13] supposes that the freeway system is global Lipschitz continuous about all iteration-varying parameters-free speed and jam density.Further, based on the norm-based theory, the proposed ILC learning law is proved convergent to track the desired traffic density despite the existence of iteration-varying parameters.It should be noted that the global Lipschitz condition is the key condition for normbased method.
Another representative method is the super-vector analysis approach.For discrete-time interval system, the supervector scheme can transform the 2D dynamic system into a 1D MIMO static plant with a Markov matrix only in iteration dimension [9].This framework provides us a unified method to design and analyze ILC for the system with model uncertainty in the iteration domain.The design problem of ILC for system with iteration-varying parameters will be converted into a Markov representation problem [14][15][16][17][18].In [16], the varying range of Markov parameters was assumed to be known, and it proved that the convergence of ILC can be guaranteed by the vertex point of the Markov parameters.Optimal method was utilized to update the control input to handle the iteration-varying parameter uncertainty in [17].The super-vector method greatly simplifies the design and analysis of ILC, and the global Lipschitz condition is no longer needed.However, due to the matrix transform, this method is not applicable for continuous-time system.
The composite energy function (CEF) is the third method.Derived from Lyapunov stability theory, the CEF method is a systematic way for learning law design and convergence analysis of the ILC of continuous-time nonlinear system without requiring global Lipschitz condition [19].For the system with periodically iteration-varying parameter, [20] incorporated adaptive control into ILC and the convergence was proved in the CEF way.The CEF analysis method has also been successfully applied to automatic train operation ILC control despite the existence of iteration-varying parameters such as temperatures [21].The CEF method does not require the global Lipschitz continuous condition, but it is restricted for continuous-time system.Due to the different applications of the above three design and analysis methods, they are complementary in the design procedure of ILC for the system with iteration-varying parameters.
In this paper, we focus on continuous-time nonlinear systems with uncertain or jumping parameters.In practice, many factors can cause the abrupt changes of system's structure and parameter, for example, faults in system, abrupt disturbances, failures of sensor and actuator, and load changes.These abrupt changes in dynamic parameters cause the operation of controlled plant in multiple environments.Traditional single model ILC can no longer handle this kind of system perfectly.Single fixed controller is established based on the unchanged model, but when parameters jump, it cannot learn from previous operation because the model is no longer the same in the iteration domain and the learning process has to start all over again.As the model is fixed and the plant is changing, mismatch between model and system is inevitable, so that the learning process gets relative long.We incorporate multiple model adaptive control (MMAC) [22,23] to cover parameter uncertainty with multiple models, detect which situation the plant is in at every iteration, and launch the corresponding ILC controller.Thus, we can improve transient response performance (in this paper, transient response performance means the iteration trails before system output converges to the desired trajectory) and control property for the system with uncertain parameters.Further, by the norm-based method, we prove that the proposed ILC method can achieve asymptotic convergence.
This paper is organized as follows.Section 2 introduces the model algorithmic ILC and formulates the dynamic system with iteration-varying jumping parameters.Section 3 presents the MMILC algorithm in detail and Section 4 analyzes its convergence for system with unknown constant parameters and system with jumping parameters.Numerical simulations are conducted in Section 5 and Section 6 concludes this work.

Model Algorithmic ILC and Problem Formulation
2.1.Model Algorithmic ILC.Consider the following continuous-time nonlinear system: where the subscript  = 0, 1, 2, . . .denotes the th iteration.u  () ∈ R  and x  () ∈ R  are accessible control input vector and system state vector, respectively, at the th iteration.
where  f and  B are the lipschitz coefficients of functions f and B, respectively, and  B is the upperbound of B.
In this paper, we denote ‖a‖ as the infinite norm of vector a ∈ R  with its entries   ; that is, ‖a‖ = ‖a‖ ∞ = max 1⩽⩽ |  |.And the notation of -norm of vector function N() ∈ R  is given as where  > 0.
For the nonlinear dynamic plant described by (1), suppose that its model is obtained as where { f, B} is the model of {f, B} and x() ∈ R  is the model state vector.Paper [24] proposed the model algorithmic ILC learning scheme: where B+ is the generalized inverse of B; that is, B+ = [ BT B] −1 BT .Suppose that B+ is bounded and B+ and B+ f are global Lipschitz continuous about x.
As the system state x  () is accessible, it is reasonable to replace x () with x  () in ILC learning law (5).If learning law (5) is applied to system (1), we have the following lemma.
Lemma 1 (see [24,25]).If the ILC system and model described by (1) and (4), respectively, satisfy (1)      I − B+ (x(), )B(x(), ) then, for arbitrarily given and achievable desired trajectory x  () ( ∈ [0, ]), ILC learning law (5) can generate a control sequence which guarantees x  () to converge to x  (); that is, The proof of Lemma 1 is attached at the end of this paper as the Appendix.Further, from the proof, we can draw the following conclusion.
As B+ and B+ f are Lipschitz continuous about x; h satisfies the Lipschitz condition; that is, for all  ∈ [0, ], and where  h is the Lipschitz coefficient of h.
For single model ILC, the proof of Lemma 1 gives that where and  is a arbitrarily given positive value.
Remark 3. From the above definition of different elements,  B ,  2 , and   2 are only related to the dynamic system (f, B), while ,  1 are decided by both dynamic system (f, B) and model ( f, B).Consequently, (8a) is related to the dynamic system while (8b) is decided by both dynamic system and model.

Remark 4.
Most ILC methods just use the information of process measurements, so corresponding control strategies are always date-based and lack mechanism analysis.In practice, a priori knowledge about the process is more or less known.The model algorithmic ILC is not a model free or a pure data-driven method; it is a model-data integrated method which also takes the estimated system model into account, so that the known information can be fully used to perfect the actual performance [26].

Problem Formulation.
In this paper, we consider nonlinear system with iteration-varying jumping parameters in the form of where   () and   () are unknown piecewise-continuous parameters which are constant at first and then change abruptly to other constant values; this change may happen at any time and in any iteration.For  = 0, 1, . . ., ∞,  ∈ [0, ], uncertain parameters vary in finite convex set Π; that is, where   and   are finite integers.The control objective is for system described as (9), to find such an appropriate control input u(), that the generated state output x() can be guaranteed to track the desired trajectory x  () as precisely and quickly as possible.In other words, the track error converges to zero as the iteration approaches infinity; that is, lim To achieve this target, we suppose that system (9) satisfies the following assumptions.
Assumption 5. Specific values of varying parameters   () and   () are unknown, but the probable varying scope is known as where   and   are the lower bound and   and   are the upper bound of  and , respectively.Assumption 6.Consider the time variant system as (9); the parameters of system will be piecewise constant, and the time between two adjacent parameter jumps is relatively long.
where   f and   B are Lipschitz parameters.
Considering Lemma 1, model algorithm ILC can somehow handle model imprecision and mismatch; however, this is based on the condition that controlled plant as (1) must be strictly repetitive along the iteration axis.For plant in the form of ( 9), the dynamic system will change as parameter jumps, model mismatch may get more serious, and the ILC method cannot learn from previous iterations and has to restart the learning process.Consequently, transient response will deteriorate and long time will be needed before the output tracks the desired trajectory for the relearning process.
To handle this problem, we introduce multiple model adaptive control to ILC.Multiple models are established to cover the parameter uncertainty, an adaptive switching scheme will be proposed to accommodate the dynamic system change, so that the transient response can be improved.

Multiple Model Iteration Learning Control
For system described by (9), the iteratively jumping parameters cause the plant to operate among multiple environments.Our goal is to detect the specific environment at current iteration, serve this environment appropriately, and finally realize fast and precise trajectory tracking in multiple environments.In this section, we discuss the establishment of multiple models according to the iteration-varying parameters, the establishment of ILC controllers, introduction of switching index function and the design of multiple model iteration learning control algorithm.

The Establishment of Multiple Models. Multiple param-
eter set P can be established manually to cover the varying scope of (, ) described as (12): where   and   are finite integers.

Assumption 9. B+
is bounded and B+   and B+   f  satisfy the Lipschitz continuous condition about x; that is, for all  ∈ [0, ], x, x 1 , where Though ILC can deal with system with model uncertainty, the more accurate the model is, the faster the transient response will be.Based on this idea, multiple models are built to cover the parametric uncertainty, so that at each iteration the most approximate model to current system will be selected and finally the performance of transience response can be improved.

The Establishment of Controllers.
For model algorithmic ILC, learning law ( 5) is based on the established model of the controlled plant.Corresponding to established multiple models, we present multiple controllers according to ILC learning law (5); that is, where Remark 10.For system (9), if parameters   () and   () finally stop jumping at a group of fixed values, every single controller   can guarantee output to converge to desired trajectory according to Lemma 1.However, the learning process has to restart as the single model may be far away from the plant when parameters change, and the process to convergent will be relatively long.
In this section, we have established multiple controllers according to the uncertainty of parameters and we hope that the most approximate model and corresponding controller will be switched into control system before every iteration so that the transient response time can be greatly reduced.Though the controller set is based on model algorithmic ILC learning law ( 5) and every single controller can guarantee system convergence according to Remark 10, how to keep the integrated system convergent with infinite switching times among multiple controllers is essential to MMILC.

Switching Index Function.
When parameters jump, original model is no longer suitable.From this perspective, the switching scheme focuses on finding the most appropriate model for current environment from model set, further activating the corresponding controller.The ultimate purpose is to improve the transient response of the adaptive system with the least prior knowledge.The switching index function is given as where e i  () denotes the error between system state and the state of model  i at the th iteration: > 0 is the forgetting factor;  > 0 and ] > 0 are parameters to adjust the weights of current and past errors in the calculation of switching index.At the end of th iteration, switching index function  i, of every model  i (i ∈ Ω) is calculated and the most suitable model for the th iteration is decided by minimum  i, ; that is, model  l(+1) will be selected if the following equation is satisfied: and l(0) = (  (0),   (0)), where   (0) = random{1, 2, . . .,   } and   (0) = random{1, 2, . . .,   }.Corresponding controller  l(+1) will be immediately selected as the controller for the next iteration.

Convergence Analysis
Convergence is the essential condition of controller design.Though every single model and corresponding controller can keep the convergence of system, how to guarantee that the entire MMILC is stable even with infinite switching among model set is the key problem for the proposed algorithm.Conditions for convergent MMILC and its detailed proof are given here.
(2) ∀, then, for any given reachable desired trajectory x  () ( ∈ [0, ]), the proposed MMILC learning algorithm will generate a control sequence, which can guarantee that the system state is convergent to the desired trajectory; that is, Proof.According to the MMILC method, multiple models From the switch index function (27) of MMILC algorithm, model  l() = ( f  () , B  () ) will be selected at the th iteration.So focusing on every single iteration, the proposed MMILC can be viewed as single model ILC.As parameters   () and   () are both constant, dynamic system is strictly iteration-time invariant, and the system Assumptions 7 and 8 can be easily guaranteed.As model Assumption 9 is also satisfied, according to Corollary 2, at the th iteration, for every single element model  i , i ∈ Ω, we have where Since   < 1, it is possible to find  > 0 large enough so that therefore, we have sup ∈[0,] ‖Δu  ()‖ → 0 as  → ∞; that is to say, u  () → u  () for  ∈ [0, ] when  → ∞.Furthermore, combing (31a), we have x  () → x  () uniformly on [0, ].

System with Unknown Jumping Parameter
Corollary 13.For iteratively operating system with jumping parameters described as (9), in addition to system Assumptions 5-8 and model Assumption 9, if the following conditions are also satisfied, (2) for all , then, for any given reachable desired trajectory x  () ( ∈ [0, ]), the proposed MMILC learning algorithm will generate a control sequence, which can improve the control property for this kind of system.
We just consider the time variant system with piecewise constant parameters.According to Assumption 6, the jumping parameters will be kept for relatively long time, and the system in two adjacent parameter jumpings can be regarded as a time-invariant system.Due to the existence of multiple model, the transient response will be improved greatly, so the state of system will converge to the desired trajectory rapidly, and the control property of piecewiseconstant jumping system will be improved greatly.

Simulation Results
Consider a single-link robotic manipulator with the following dynamic system: which operates iteratively on the fixed time interval; that is,  ∈ [0, 10], and  denotes the iterations., θ , and θ are the joint angle, angular velocity, and accelerated velocity, respectively. = 0.75 2 kg ⋅ m 2 is the moment of inertia of the manipulator,  = 1.25 m is the length of the manipulator,  = 1.0 kg is the mass of the joint,  = 9.8 m/s 2 is the gravity,   () N ⋅ m ⋅ s/rad is the viscous friction coefficient which is decided by the shape and size of the link and the viscidity of the lubricant, and   () is unknown parameter varying in [0, 5].  is the control input.
As an illustrative example, there is jumping parameter only in the f part; the following can be verified: so it can been seen apparently that the system and established multiple models satisfy all the Lipschitz and norm bounded conditions.So the proposed MMILC algorithm can be applied to the controlled system according to Theorem 12 and Corollary 13.   1(c).However, when there is relative great mismatch between model parameter d = 1/6 and plant parameter   () = 33/7, it takes more than 30 iterations before the desired state output is obtained as shown in Figure 2.

Multiple Model ILC.
In this part, when unknown system parameter is   () = 33/7, multiple models are established as (40), and the multiple model ILC control Algorithm 11 is applied to the plant, control result can be found in Figure 3.
As in Figure 2, when there is relatively great mismatch between model parameter and system parameter, single model ILC cannot track the desired trajectory within reasonable iterations.However, in multiple model ILC control scheme, multiple models are established to cover the parameter's varying scope; in Figure 3(b), MMILC can identify that  1 with  1 = 1/6 is not the best one for plant, and the most appropriate model  6 with model parameter  6 = 31/6 is selected immediately since the second iteration.Since the introduced multiple model set and switching mechanism, iterations are decreased greatly to about 6 as shown in Figure 3(a) compared with single model ILC with d = 1/6 shown in Figure 2.

Plant with Jumping
Parameter.For plant (37), if   () is a piece-wise constant jumping parameter in the form This parameter jumping process is shown as in Figure 4(b) in which the length of each iteration is equal to the fixed time interval.
If single model ILC with parameter d = 1/6 and multiple model ILC with multiple models as (40) are adopted to control the plant, respectively, state errors respective to each iteration of these two methods are shown in Figure 4(a).
When parameter jumping happens, as the dynamic system will change at the same time, no matter single model or multiple model ILC, the learning process has to restart.In the first 30 iterations, d = 1/6 in single model ILC is close to system parameter () = 3/7, so the control effect looks as well as the effect of MMILC.However, for single model ILC, when the parameter deviates far from model parameter, the approaching process gets relatively long, as in Figure 4(a); it takes about more than 30 and 20 iterations before the state   output converges to the desired track after the second and third parameter jumpings.
According to the MMILC strategy, multiple models are established to cover the parameter uncertainty; at the end of each iteration, the index functions showing the matching degree between each model and the current plant are calculated.In the next iteration, the closest matching model and corresponding controller will be selected.The parameter jumping and model switching process are shown in Figure 4(b).In the 30th iteration, there is a parameter jumping, consequently, model  6 is selected at the 31st iteration; parameter change in the 60th iteration is also identified and the most closest model  4 is switched to system.Taking this model switching scheme, convergent processes after each parameter jumping are reduced greatly to about 5 iterations as shown in Figure 4(a).

Conclusion
In this paper, we have incorporated multiple model adaptive control into model algorithm iterative learning control for nonlinear repetitive operating system with jumping parameters.In order to reduce the iterations before the control state converges to the desired trajectory when there is model mismatch, multiple models are set up to cover the parameter uncertainty; model index function has been established
Parameter jumping and model switching process

Figure 4 :
Figure 4: Control results of single and multiple model ILC dealing with plant with jumping parameter.
is the bound of B+   , and    and  i are the Lipschitz coefficients of B+   and B+ Model ILC Control Algorithm.Based on the above discussion, the main procedures for multiple model ILC control algorithm are provided.According to varying scope of   () and   (), multiple model set will be established as follows: (9).System with Unknown Constant Parameter.For system(9), if parameters   () and   () are unknown constant values irrelative with  and , we have the following theorem.For iteratively operating system depicted as(9)but with unknown constant parameters   () and   () which are irrelative with  and , if Assumptions 7-9 and the following conditions are satisfied, (x, )[ ẋ  () − f  (x, )].As B+   and B+   f  are Lipschitz continuous about x by Assumption 8, h i satisfies the Lipschitz condition; that is, ∀x 1 will be established based on the parameters' varying scope.For notational simplify, we denote L   (x, ) = B+   (x, ) and h i (x, ) = L 2,...,  }    , and   1 =   h +   L   2 , for arbitrary model switching among the model set , (31b) can be written as     Δu +1     ⩽ (  +   1 B    2 )    Δu      .
(5)nown Time-Invariant Parameter.If plant (37) contains unknown time-invariant parameter, control results by using single model ILC algorithm(5)and proposed multiple model ILC Algorithm 11 can be obtained as follows.