Adaptive Control Using Fully Online Sequential-Extreme Learning Machine and a Case Study on Engine Air-Fuel Ratio Regulation

Most adaptive neural control schemes are based on stochastic gradient-descent backpropagation (SGBP), which suffers from local minima problem. Although the recently proposed regularized online sequential-extreme learning machine (ReOS-ELM) can overcome this issue, it requires a batch of representative initial training data to construct a base model before online learning. The initial data is usually difficult to collect in adaptive control applications. Therefore, this paper proposes an improved version of ReOS-ELM, entitled fully online sequential-extreme learning machine (FOS-ELM).While retaining the advantages of ReOS-ELM, FOS-ELM discards the initial training phase, and hence becomes suitable for adaptive control applications. To demonstrate its effectiveness, FOS-ELM was applied to the adaptive control of engine air-fuel ratio based on a simulated engine model. Besides, controller parameters were also analyzed, in which it is found that large hidden node number with small regularization parameter leads to the best performance. A comparison among FOS-ELM and SGBP was also conducted. The result indicates that FOS-ELM achieves better tracking and convergence performance than SGBP, since FOS-ELM tends to learn the unknown engine model globally whereas SGBP tends to “forget” what it has learnt. This implies that FOS-ELM is more preferable for adaptive control applications.


Introduction
Adaptive control is a powerful control scheme for dynamic system with high uncertainty.Its principle is to, based on the output feedback of the system, self-adjust the characteristics of the controller online in a way that the tracking error is reduced while stability is maintained.One remarkable development in adaptive control is the application of neural networks to the adaptive mechanism [1][2][3], which is often referred to as adaptive neural control.It is well known that neural networks can approximate any nonlinear relationship by means of different network parameters and activation functions.Therefore, by expressing the system uncertainty in terms of neural networks, an adaptive neural controller is able to handle arbitrary nonlinearities through the tuning of its unknown network parameters.With this attractive feature, adaptive neural control has been extensively used in many controller design problems and practical applications [4][5][6][7][8].
Nevertheless, in most typical neural controllers, the parameter adjustment method, or so-called the adaptive law, is based on the backpropagation (BP) algorithm [9].The critical drawback of this algorithm is that it is a gradientdecent based learning method which may easily converge to local minima [10,11].Therefore, it usually takes "more than required" steps for the controller to achieve satisfactory performance.For instance, the simulation results in an earlier work of adaptive neural control [1] showed that thousands of updating steps were needed before the controller could finally achieve the desired convergence.Moreover, in some recent studies such as [5,6] the neural controllers were shown to perform better than traditional proportional-integralderivative controllers.However, the controllers still take many time steps to settle every time when the desired output is changed.These results indicate that the system dynamics cannot be globally approximated.Another disadvantageous property of BP is that it updates the parameters in all the layers of the neural network, leading to a long processing time and hence a slow convergence speed.
In order to address the issues of BP, Huang et al. [12,13] proposed a simple and fast algorithm entitled extreme learning machine (ELM).This algorithm trains only a single hidden layer feedforward neural network.Unlike BP where all the parameters need to be tuned, ELM learns the unknown nonlinear relation by updating only the output weights (parameters between the hidden layer and the output layer of the neural network); the parameters in the hidden layer are randomly initialized and remain unchanged.Due to its simple structure and learning mechanism, ELM runs much faster (up to thousands of times [11][12][13]) than traditional BP.Meanwhile, ELM is also superior to BP in terms of generalization performance and accuracy, which has been verified in many latest works [11,[14][15][16][17][18].In this sense, employing ELM into adaptive neural control should lead to a better control performance.Yet the original ELM algorithm is only suitable for batch learning.To learn the model online, online sequential ELM (OS-ELM) was proposed in [19].While achieving the same performance as batch ELM, OS-ELM could update the network parameters sequentially no matter whether the data comes one by one or chunk by chunk.Therefore, by replacing BP with OS-ELM, a better and faster adaptive neural controller should be obtained.
However, there are some factors limiting the direct application of OS-ELM to adaptive control.Firstly, OS-ELM is not robust for noisy data.Secondly, the initial parameters of OS-ELM, which are randomly generated, can easily lead to singular and ill-posed problems [20].These problems significantly affect the model, so that the generalization performance could degrade to an unacceptable level.Furthermore, theoretically speaking, OS-ELM is not a fully online sequential learning algorithm; it requires a chunk of representative initial data to train a base ELM model in advance to the online sequential learning.This chunk of representative initial data is usually difficult to obtain for adaptive control problems.The number of the initial data could not be less than the number of hidden nodes either.All these together highly restrict the use of OS-ELM.
In order to deal with the aforesaid problems, regularized OS-ELM (ReOS-ELM), which was proposed by Huynh and Won [20], could be used.In ReOS-ELM, the norm of output weights is added to the objective function to avoid singular and ill-posed problems.At the same time, a regularization parameter is included for the trade-off between the optimization of output weight norm and the training error.With the introduction of the regularization parameter, the number of training data could also be less than the number of hidden nodes.A base model, however, is still required in ReOS-ELM.To overcome this limitation, this paper proposes a fully online version of ELM, entitled fully online sequential-extreme learning machine (FOS-ELM).This proposed FOS-ELM is derived from ReOS-ELM, so it retains all the advantages and properties of ReOS-ELM, with the only difference that the batch training phase is discarded.Due to the removal of batch training phase, FOS-ELM can easily be applied to any adaptive control problems.For demonstration purpose, this paper presents the application of FOS-ELM to the adaptive engine air-fuel ratio (AFR) control based on a simulated engine model.The influence of the parameters (regularization parameter and hidden node number) is analyzed in the simulation.To verify the effectiveness of FOS-ELM, stochastic gradient-descent BP (SGBP), as a sequential learning variant of BP, is also applied to the same adaptive AFR control problem for comparison.
The organization of this paper is as follows.A brief review of ELM and its variants is provided in Section 2. The details of the proposed FOS-ELM are presented in Section 3. The application of FOS-ELM to adaptive engine AFR control and the related discussions are given in Section 4. Finally, conclusions are drawn in Section 5.

Review of ELM and Its Variants
This section briefly reviews the related work of ELM, including basic ELM, regularized ELM (ReELM), OS-ELM, and ReOS-ELM, in order to provide necessary background.[13] is an emerging technique for training feedforward neural networks without iterations.It consists of only one hidden layer, in which the input weights are randomly generated and need not be tuned.The output weights are optimized using a Moore-Penrose pseudoinverse instead of gradient-decent method.Apart from the number of hidden nodes, no other parameters have to be manually chosen [13,19].For a network with one hidden layer and  hidden nodes, the output function is

ELM and Regularized ELM. ELM
where h(x) = [ℎ 1 (x), ℎ 2 (x), . . ., ℎ  (x)] is the output vector of the hidden layer feature mapping with respect to the input x and  = [ 1 ,  2 , . . .,   ]  is the vector of output weights between the hidden layer and the output nodes.For a training dataset  with  samples, matrix H = [h(x 1 ), h(x 2 ), . . ., h(x  )]  can be used to present the hidden layer output.The size of H is  ×  and each row of H is a training sample after feature mapping.
The goal of basic ELM is to minimize the training error; that is, where T is the vector of real target   with respect to a sample   from .Mathematically, it is a multiple linear regression problem.The solution of  to (2) is where H † is the Moore-Penrose generalized inverse of matrix H.If H  H is nonsingular, the orthogonal projection method can be used to calculate the pseudoinverse of H Thus,  can be rewritten as Since basic ELM is based on empirical risk minimization principle (please refer to (2)), the trained model tends to be overfitting [20,21].Therefore, ReELM was proposed in [21] as an improved version of ELM.A similar work has also been introduced by the authors of ELM in [22], and a more detailed explanation can be found in [23].The optimal goal of ReELM is to minimize not only the training error, but also the norm of the output weights; that is, The optimization problem of ReELM for a single-output node can then be formulated as follows: Minimize: where  is the user-specified parameter that provides a tradeoff between the training error and the norm of the output weights,  is the number of training data, and   is the error for th training data (also known as slack variable).The solution of  can be calculated by According to Bartlett's theory [24], this resulting solution tends to have better and more stable generalization performance, as verified in [21][22][23].

OS-ELM and ReOS-ELM.
OS-ELM, originated from basic ELM, is an online sequential learning algorithm that can learn data not only one-by-one but also chunk-by-chunk with fixed or varying chunk size [19].It consists of two phases: initialization phase and sequential learning phase.In the initialization phase, a base ELM model is trained using a small chunk of initial training data.For instance, the output weight for an initial training dataset  0 with  0 training samples is obtained as Then, in the sequential learning phase, when a new chunk of training data arrives, the output weights are updated by where  + 1 indicates the ( + 1)th arriving training data with  starting from zero and H +1 is the hidden layer output for the ( + 1)th arriving training data.
One major problem in OS-ELM is that, if the term H  0 H 0 is singular, then (10) is unsolvable.Therefore, to avoid the singular problem, OS-ELM restricts that the initial training dataset  0 should have at least  (hidden node number) distinct samples.To improve this situation, ReOS-ELM [20] adds a regularization term to (10); that is, According to the ridge regression theory, adding a small positive value into the diagonal of H  0 H 0 can also avoid singular problem when the number of initial training data is less than the hidden nodes number.Therefore, ReOS-ELM can resolve the constraint suffered in OS-ELM, making it suitable for case when initial number of data is small (e.g., adaptive control problems).In addition, similar to ReELM, the term  in (13) of ReOS-ELM mainly controls the relative importance between the training error and the norm of output weights.The theory behind the improvement of ReOS-ELM over OS-ELM can be explained using the same reason of ReELM over basic ELM.

Proposed FOS-ELM
In this section, an improved ReOS-ELM, namely, fully online sequential-extreme learning machine (FOS-ELM), is proposed.It does not need a small chunk of initial training data to construct a base model but can achieve the same performance with ReOS-ELM.
Considering an initial training dataset  0 = {(  ,   ) |  = 1, . . .,  0 } with a corresponding hidden layer output matrix H 0 , using ( 9) and ( 13), the output weights  0 are calculated as where arrives with a corresponding hidden layer output matrix H 1 .By considering both training datasets  0 and  1 , using ( 9) and ( 13) again, the output weights  1 should be obtained as where .Now expanding the last two terms on the right-hand side of ( 15) Then, combining (15), (16), and ( 17),  1 is obtained as Now, considering only  1 ,  1 can be obtained as Comparing ( 18) and ( 19), it is obvious that ( 19) can be obtained from (18) if and only if  0 = 0 and K 0 = I.Therefore, by initializing  0 = 0 and K 0 = I, the initial training datasets  0 can be omitted, while a model for  1 can still be constructed.In other words, the batch training in the initialization phase of ReOS-ELM is automatically integrated in FOS-ELM.Thereby, FOS-ELM becomes a fully online sequential learning algorithm and still can achieve the same learning performance with ReOS-ELM.
To make it clear, the proposed FOS-ELM algorithm is rewritten as below.
In short, FOS-ELM is a fully online sequential learning algorithm.It is simpler than ReOS-ELM and easier to implement.Compared with OS-ELM and ReOS-ELM, FOS-ELM is more suitable for learning problems in which the training data is difficult to collect in advance.To emphasize the advantages of FOS-ELM, a detailed comparison among OS-ELM, ReOS-ELM, and FOS-ELM is summarized in Table 1.
As declared in [19], the sequential learning algorithm (11) and ( 12) of OS-ELM is similar to recursive least-squares (RLS) algorithm, so that all the convergence results of RLS can be applied.It has to be noted that, in fact, ReOS-ELM and FOS-ELM also share the same sequential learning update algorithm with OS-ELM, so the convergence results of RLS can also be applied to all the three algorithms, OS-ELM, ReOS-ELM, and FOS-ELM.In other words, if the three algorithms are applied to the adaptive controller, the controller stability can be guaranteed.

Case Study on Adaptive Engine AFR Control
To demonstrate the usefulness of the proposed algorithm, FOS-ELM is applied to the adaptive control of engine AFR based on a simulated engine model.The effectiveness of FOS-ELM in this application is discussed and a comparison with SGBP is provided in this section.

Adaptive Engine AFR Control.
Engine AFR refers to the mass ratio of air to fuel present in the engine.It is a parameter that critically affects the engine emissions, brakespecific fuel consumption, and power [25].In general, the AFR can be set to different values for different purpose.For example, using gasoline as the fuel, the AFR should be controlled to the stoichiometric AFR of gasoline, 14.7 : 1, in order to keep maximum conversion efficiency of the threeway catalytic converter [25].In case higher engine torque is demanded, the AFR should be controlled to 12.5 : 1 in order to achieve the best engine power.For the best brake-specific fuel consumption, the AFR should be set to 16 : 1.Consequently, controlling the AFR is essential for maintaining the desired engine performance.However, the combustion process of an engine is a complex dynamic system that involves many uncertainties [11,15].Therefore, for illustrative purpose, this paper applies the adaptive control scheme, based on the proposed FOS-ELM, to the AFR control.
Theoretically, the dynamics of AFR can be described by a discrete approximated model in which the control appears linearly [1]: where  is the AFR,  is the control input,  is the time step,  is the system order, and (⋅) must be a nonzero function.If both (⋅) and (⋅) are known, the following control law can be used to exactly track the desired AFR,   : Therefore, assuming that FOS-ELM consists of two functions ĝ(⋅) and φ(⋅), the purpose of FOS-ELM is to adaptively learn (⋅) and (⋅) by self-tuning the parameters of ĝ(⋅) and φ(⋅), based on the error from the system output feedback (i.e., ĝ(⋅) → (⋅) and φ(⋅) → (⋅)).The engine AFR control scheme is illustrated in Figure 1.
The purpose of the controller is to control the amount of fuel injected to the engine so that the corresponding AFR can match the target AFR.The control signal  in (21) is the fuel injection time of the injectors.The longer the fuel injection time is, the larger the amount of the fuel injected is.To simplify the problem and focus on performance of FOS-ELM, a simulated engine model (465Q gasoline engine at  engine speed of 3500 rpm and manifold pressure of 85 kPa) [26] is used in this paper, given as Two reference AFR outputs (  ) are used to evaluate the performance of FOS-ELM.The first one is a square wave, of which the amplitude changes between 12.5 and 14.7 every 50 steps.This can test the step response of the adaptive controller.The other reference command is a sine wave, of which the amplitude varies between 12.5 and 14.7 with a period of 100 steps.This, on the other hand, can test the continuous tracking performance of the controller.In addition, all the simulations in the following sections were implemented in MATLAB and executed on a PC with Intel Core i7 CPU and 4 GB RAM onboard.

Performance of FOS-ELM.
The performance of FOS-ELM on the adaptive AFR controller is evaluated by three cases.As compared to OS-ELM, there is a regularization parameter  introduced in FOS-ELM.Therefore, the first case is to test the effectiveness of .Moreover, as there is another important parameter in FOS-ELM, namely, the hidden nodes number , the influence of these two parameters on the performance of FOS-ELM should also be analyzed.Thus, in the second case, simulations under various  and  values are presented.In the last case, disturbances are introduced to the reference command in order to test the robustness of FOS-ELM.

Effectiveness of 𝜆.
In this case, the FOS-ELM used in the adaptive AFR controller was run under two  values:  = 0 and  = 0.001.For both situations,  was set to 30.The simulation results are presented in Figure 2.
It can be seen that the tracking performance of FOS-ELM without using  is quite poor as compared to that with .This is mainly due to the singular problem.It should be noted that when  = 0, it becomes a special case of OS-ELM which trains the base model using the first arriving training sample.As in this adaptive problem, the training sample arrives in a one-by-one manner; the number of initial training data in the first step is also one.This is against the restriction of OS-ELM that the initial training dataset should have at least  distinct samples in the initial phase.Thus, the term H  0 H 0 is mainly determined by the first arriving training sample and the singular H  0 H 0 is inevitable.
Mathematical Problems in Engineering Furthermore, the rank of P  (used in the update phase as given in (12)) basically remains at one in the sequential learning phase, so the learning performance of the FOS-ELM without  is simply equal to an OS-ELM with only one hidden node; even a large number of hidden nodes are set.As a result, the tracking error of FOS-ELM without using  cannot be eliminated.

Influence of 𝜆 and 𝐿.
Referring to (7), the regularization parameter  mainly controls the relative importance of the two terms: training error and norm of output weights.
If  is small, reducing the training error is more important.Otherwise, minimizing the norm of output weights is more important.Therefore, by setting  to a small value, the tracking error will rapidly be reduced.However, if  is too small (very close to 0), the matrix (H  H + I) will tend to become singular, which leads to the situation suffered in the first case.Moreover, the hidden node number  is another factor that can affect the performance of FOS-ELM.It mainly controls the dimensionality of the model.Since, in all the variants of ELM, the parameters in the hidden node are randomly generated, it should follow that the larger the hidden node number is, the better the representation power is.As  is already introduced in the FOS-ELM model, the overfitting problem due to large hidden node number is avoided.To analyze the influence of  and  on the model, simulations were run at 4 different  values (10 −7 , 10 −5 , 10 −3 , and 10 −1 ) and 3 different  values (10, 30, and 50).The norm of the output weights of the FOS-ELM at each step was recorded.The results are shown in Figure 3.The results in Figure 3 show that the influence of  is more significant than .As shown in Figure 3(a), when  is small, the norm of the output weights tends to "blow up, " especially when  is close to zero (e.g.,  = 10 −7 ).This "blow up" often leads to bad generalization performance [24].On the contrary, as shown in Figure 3(c), if  is set to a large number, say 50, then different  values do not affect the norm of output weights too much; the trend at different  basically remains the same.This indicates that  is not very sensitive to the norm of output weight, as long as it is not zero and  is sufficient.In order to investigate how the change of  affects the controller performance, simulations were run at two different  values:  = 10 and  = 50, with a fixed  = 0.001.The results are shown in Figure 4.
From Figure 4, it can be seen that a FOS-ELM with a larger number of hidden nodes has better tracking performance.This verifies the idea that the hidden node number is in proportion to the representation power.As a remark, these simulation results are in accordance with the proof of ELM by Huang et al. [13].

Robustness of FOS-ELM.
One powerful feature of adaptive controller is its robustness to disturbance.Therefore, to evaluate the robustness of FOS-ELM on this adaptive control application, two disturbances were introduced to the reference AFR command at  = 120 and  = 180.The simulations were run at  = 0.001 and  = 50.The results are presented in Figure 5.It shows that although disturbances are introduced, the FOS-ELM adaptive controller can converge quickly back to the reference command, indicating that FOS-ELM is quite robust to disturbance.[9], is a typical algorithm for adaptive neural control.In order to show the benefits of FOS-ELM over SGBP on adaptive neural control, SGBP was also applied to the adaptive engine AFR control problem, and a comparison between FOS-ELM and SGBP was carried out.Similar to FOS-ELM, SGBP has two parameters, known as learning rate and the hidden node number.In the comparison, the hidden node number for SBGP is 20, and three learning rates ( = 0.005, 0.01, 0.05) were assigned.For FOS-ELM, the hidden node number is set to 50, with a regularization factor of 0.001 again.The results of the two methods are provided in Figure 6.

Comparison to SGBP. SGBP, being a variant of BP for sequential learning application
The comparative results from Figure 6 imply two things.The first one is that SGBP is quite sensitive to its learning rate.As shown in Figure 6(a), a small learning rate leads to a slow convergence speed, while, as shown in Figure 6(c), a large learning rate leads to an oscillation in the convergence process.It was found from some preliminary tests (not shown here) that the learning rate is also strongly associated with the hidden node number.In other words, to determine the structure and parameters of SGBP, expert experience is necessary [11].Comparing to SGBP, FOS-ELM is less sensitive to parameters.Usually, the regularized parameter  can be set to a small value like  = 0.001, and the hidden node number  can be set to a large value like 50.This has already been verified in the previous section.
Another implication from Figure 6 is that FOS-ELM can achieve better global control performance as compared to SGBP.Referring to Figure 6(b), every time when the amplitude changes, several steps were required by SGBP to adapt to the desired reference.This shows that SGBP always tends to "forget" what it has learnt.The reason behind this phenomenon is that SGBP updates both input weights and output weights to achieve the desired output, which may easily suffer from local minima (i.e., optimal for the most recent arrived data).Thus, when the desired output changes, the weights need to be adjusted again for tracking the desired output.In contrast, the theory behind FOS-ELM is to seek for a global optimal (i.e., optimal for all the seen data).Hence, as shown in Figure 6(d), once the model is learnt, the controller can directly adapt to the desired output even if it changes frequently.In fact, by referring to Figure 3, the norm of the output weights becomes stable after several learning steps, which also verifies that the model has been learnt and no further update is required.This unique feature of FOS-ELM is highly suitable for adaptive control applications.

Conclusions
In this paper, a novel fully online learning algorithm entitled FOS-ELM is proposed for adaptive neural control.It keeps the same learning performance with ReOS-ELM but discards the initial batch training phase adopted in ReOS-ELM.Without the initial training phase, FOS-ELM becomes easier to be implemented and more suitable for online learning task, of which the training data is difficult to be provided in advance, for example, adaptive control problems.
To demonstrate its effectiveness, the proposed FOS-ELM is applied to the adaptive control of engine AFR based on a simulated engine model.As the performance of FOS-ELM is determined by two important parameters, namely, regularization parameter and hidden node number, the influence of these parameters was analyzed.Furthermore, a comparison between FOS-ELM and SGBP on the adaptive control application was also carried out.The results imply the following.

Figure 2 :
Figure 2: Effectiveness of  for (a) square wave reference; (b) sine wave reference.

Figure 4 :
Figure 4: Influence of  for (a) square wave reference; (b) sine wave reference.

Figure 5 :
Figure 5: Effect of disturbance for (a) square wave reference; (b) sine wave reference.