Adaptive Predictive Control: A Data-Driven Closed-Loop Subspace Identification Approach

,


Introduction
With the development of industrial technology, the industrial processes become more complex than before and it is more difficult to build the accurate mechanism models of these processes. Hence, the data-driven approach has obtained widespread attention since it emerged. Data-driven control also turns into focus of study. Simply, the data-driven control is a method from data to design controller directly [1,2]. Model predictive control (MPC) has been attractive for decades in control theory field. It has become more established as the one of the choices for the control architecture in the industry, especially with the improvement of computational capabilities of processors [3][4][5][6][7][8]. But one drawback of the traditional industrial predictive control is based on inputoutput model, including parametric and nonparametric ones. In order to improve the control performance, a state-space model should be adopted, so the modern filter theory and the design method of controller developed in recent years can play a role [9]. Subspace identification is one of the system identification algorithms for state-space modeling. The control workers may relieve completely from the tedious mechanism modeling and the accurate state-space model can be obtained when there is enough process input-output data [10][11][12]. More attractively, the subspace matrices obtained through the subspace identification algorithm can be used to derive the predictor of predictive controllers, eliminating the intermediate step of process model identification and providing a method of data-driven predictive control [13]. This method has been applied in some industrial processes and achieved good results.
Most data-driven predictive controllers are designed based on open-loop subspace identification, but in practice it is often necessary to perform identification experiments on systems operating in closed-loop. This is especially true when open-loop experiments are not allowed due to safety (unstable processes) or production (undesirable open-loop behavior) reasons [14]. It is found that the regular open-loop subspace identification algorithm yields a biased estimate when applied to closed-loop data [15]. The closed-loop datadriven predictive control methods in [16,17] have been presented. But the predictor is derived with the estimated Markov parameters which lead to a complicated predictor. We get a simple predictor constructed by subspace matrices. Jansson [18] developed a subspace method that can perform well on data collected both in open-and closed-loop conditions.
It is a major problem to implement adaptive control in closed-loop system. In this paper, based on the subspace prediction model derived from [18], we design a closed-loop data-driven predictive controller to solve this problem that obtains subspace matrices simply from Hankel matrices for a better implementation of the following adaptive mechanism in closed-loop system.
The control performance of predictive control is dependent on the model quality [19]. The linear fixed model is used to design the controller in conventional data-driven predictive control method. It is applied to a linear system showed good results in a short period. But there are nonlinear and time-varying characteristics of long period in industrial processes, resulting in a poor performance when using the fixed model. It is highly desirable to implement adaptive mechanism to adjust the system model online. The feature of subspace identification is suitable for designing adaptive predictive controller perfectly. The adaptive mechanism is realized by online updating subspace matrices. At present, there are two ways of online adaptive subspace identification [20]. One is recursive identification method; by using different weighting to the new and old data, the variation of the process is tracked. The size of modeling data set will become larger with the process operation which needs enough memory storage. The other one is receding window method; the size of modeling data set remains unchanged and the oldest data is removed at the arriving of the new data. It is unfavorable that the harmless (or useless) data will increase information missing in the whole window and the computation time is longer than recursive one [21]. The recursive adaptive predictive control method is shown in [22,23]; in [22] an adaptive predictive control strategy based on recursive subspace identification has been presented, adopting the prediction model with the smallest matching error. Mardi and Wang [23] presented an approach to constrained subspace-based MPC of time-varying systems. The central ideas are to find the predictive control law recursively using a subspace identification technology and to update the control law once a plant-model mismatch is detected. Although both of them consider the forgetting factor to weaken the negative impact of the old data on the identification model, the identification accuracy will be declined as the old data more or less. Accordingly, we can find the receding window method in [24,25]. Yang and Li [24] designed a subspacebased predictive controller, using receding window method to update subspace matrices at each time step for adaptive mechanism. Wahab et al. [25] proposed a direct adaptive MPC method which requires a single QR decomposition for obtaining the controller parameters and uses a receding horizon approach to process input-output data for the identification. These two methods require decomposition at every time instant which increase the computational load and have incapability of handling harmless (or useless) data that bring performance degradation. Only one way of online adaptive subspace identification is employed in the above adaptive predictive control methods. We have been trying to combine the two ways, in our previous work [26]; an adaptive mechanism through online updating of the matrix is proposed. By comparing the prediction error before and after updating, we consider whether or not to update the prediction model. This method employs a recursive strategy to derive matrix but it requires us to compute every element value of matrix that increases the computation time. The model inspection can bring a promotion in harmless (or useless) data suppression but it cannot eliminate the harmless (or useless) data. Kameyama et al. [27] derived a recursive subspace-based identification algorithm with fixed inputoutput data size. It only solves the identification problem. We get the online updated subspace matrices from partial results in [27] but stress the derivation of the key elements of matrix which can reduce the computation time compared to the method in [26] and extend it to design the predictive controller. Another major problem to implement adaptive control is the inaccuracy of online excitation data. When the model or system parameters change, it needs to be adequately excited. Otherwise, some of the obtained data become harmless (or useless) ones which have a negative impact on system performance. The data inspection strategy introduced is a good solution for this problem through comparing the prediction error.
The main contribution of the paper is the development of a new solution of data-driven adaptive predictive control ensuring adaptation of closed-loop systems. The method can offer an attractive alternative for industrial nonlinear, timevarying systems of long period in closed-loop condition and there is no need for obtaining the system explicit model which can reduce the complexity. Through transforming system model form, the closed-loop subspace identification algorithm is developed and the subspace matrices are obtained from the closed-loop data. The adaptive mechanism is implemented by combining the advantages of receding window and recursive identification methods. The subspace matrices are derived by recursive method using a fixed modest size of data set with receding window method. The proposed mechanism can sufficiently fade the influence of the old data better than only recursive method and bring less computation load than only receding window method. By comparing the prediction error before and after updating, we consider whether or not to add the new data in data inspection strategy. The purpose of the strategy is to eliminate the new arrival of harmful (or useless) data produced by the online insufficient excitation. The control performance is superior to adopt open-loop identification and other methods of data-driven adaptive predictive control.
The paper is organized as follows. In Section 2 the open-loop data-driven predictive control method is given. Section 3 provides the closed-loop data-driven predictive control method. The adaptive mechanism is highlighted in Section 4. Some simulation results are presented and discussed in Section 5. Section 6 gives the conclusions.

Open-Loop Data-Driven Predictive Control
Consider a discrete state-space system of order described by innovations form where ∈ R , ∈ R , and ∈ R are input, output, and state vectors, respectively. is the Kalman filter gain and ∈ R is an innovation sequence where variance ( ) = . ( , , , ) are system matrices of appropriate dimensions and is the innovations covariance matrix.
Construct the inputs block Hankel matrices using the data of with ∈ {1, 2, . . . , } at instant : where the subscripts and represent the "past" and "future" time. Similarly, the outputs and noise Hankel matrices , , , and can also be obtained in the same way. The system past and future state sequences are defined as The subspace prediction expression of the outputs can be derived by recursive substitution of (1): where Γ ∈ R × is the extended observability matrix and ∈ R × and ∈ R × are the low triangular Toeplitz matrices, respectively, denoted by ] .
The optimal prediction of can be written aŝ where denotes the past input-output data matrix as is the subspace matrix that corresponds to the past input-output data, and is the subspace matrix that corresponds to the future input data.
In order to calculate the subspace matrices and from block Hankel matrices, by solving the following least squares problem: where ‖ ⋅ ‖ represents the Frobenius norm, the solution can be found from the orthogonal projection of the row space of onto the row space of the matrix ( ): where / denotes the orthogonal projection. The solution for (8) can be done in an efficient way by performing a decomposition: where is a low triangular matrix and is an orthogonal matrix. By letting where superscript † represents the Moore-Penrose pseudoinverse and ∈ R × ( + ) , ∈ R × . The model predictive control problem is realized by the minimization of a cost function. A typical form of cost function in MPC is given as follows: where is the reference setpoint signal at the current time , and are the weight matrices, and and are the prediction and control horizon, respectively. and are defined as being equal to , and (12) can be rewritten as In MPC framework, only the leftmost column is used to predict output. And to avoid steady-state error, the predictor of predictive controllers can be written in terms of incremental Δ and Δ as follows: where Using (14) in the minimization of cost function of (13), the control sequence can be obtained as follows: At each time instance, only the first element of Δ is used for calculating the control input. Therefore the control input is drawn as At the next instant, when the new input-output data arrive, the same optimization is repeated. The above results can also be seen in [28][29][30][31][32]. In the above objectives, subspace matrices are identified using the open-loop data and applied to the open-loop system suitably. But, in closed-loop system, as the data correlations due to feedback, above identification algorithm will result in a less accurate model and it will lead to degradation in control performance. To overcome the drawback, a closed-loop data-driven predictive control method is given in Section 3.

Closed-Loop Data-Driven Predictive Control
The structure of closed-loop data-driven predictive control method is shown in Figure 1.
In order to use the closed-loop structure of the subspace identification technique, the necessary steps are presented. Firstly, transform the system model in (1) It is well known that we can rewrite system model form as follows: The prediction model can be represented as the subspace expression: wherẽ ] .
Next, it's directly to obtain the system state-space model in previous paper [18]. But in this paper, we focus on the derivation of subspace matrices to implement data-driven predictive control. Equation (20) can be rewritten as wherẽ= ( −̃) and is the appropriate identity matrix.̃can be denoted by constituting the subspace matrices as follows: The intermediate subspace matrices̃and̃are provided by the least squares problem: The solution procedure is similar to the derivation of and in Section 2. Therefore, the closed-loop subspace matrices and can be calculated as Abstract and Applied Analysis 5 We use incremental form to denote the predictor: So the control sequence becomes where 1 is an identity matrix of size 1. The control input is At the next time step, measuring the new input-output data and the new control input will be calculated using the above optimization.
The above method relies on transforming system model form for reducing the impact of the noise sequence on input sequence greatly. It can be applied in closed-loop system but also is suitable for open-loop system.

Adaptive Mechanism
The linear fixed model is used to design the controller in traditional data-driven predictive control. But, in industrial processes, in presence of nonlinear and time-varying characteristics, the control performance is difficult to achieve the desired control effect and it will cause great mismatch of the model. Therefore, the adaptive control methods, updating the model online according to the conditions, have been attractive for decades and gradually applied to industrial processes. The adaptive predictive control, one of the adaptive methods, also has achieved a number of applications [33]. In this paper, an adaptive predictive control method is presented. Drawing the advantages of the receding window approach, the size of window is maintained as modest a priori while the recursive approach is used for updating the model. Additionally, due to the system disturbance and noise, a larger match error will be produced between the test data with the real time data at some time when the model or system parameters change. Such data is referred to as the harmful (or useless) data. A data inspection strategy is suggested to use the 1-step output prediction error for filtering the harmful (or useless) data and eliminating the negative impact on the system of the harmful (or useless) data. Then, updating the subspace matrices online and implementing the adaptive mechanism are done.
The subspace matrices are obtained from matrix, so we update the matrix online using recursive method; then the prediction model can be obtained to calculate the control input.
Let * ∈ R 2( + )( + )×( − − +1) be the input-output Hankel matrix at instant as where ( ), ( ), and ( ) are the past input-output data matrix, future input data matrix, and future output data matrix, respectively, in closed-loop system. The oldest column of * is defined as Given a set of new input-output data = The input-output Hankel matrix * at instant + 1 is defined as where ( + 1), ( + 1), and ( + 1) are similar to the definitions of ( ), ( ), and ( ). In order to maintain the size of receding window constant, it is necessary to exclude from * and add to * . ] .
The objective is to get the results from the decomposition of * : * = ( + 1) ] .  ] .
From (33), firstly, we can get the first element 11 ( + 1) of ( + 1): where chol is Cholesky factorization [34]. The subspace matrices are obtained from matrix as in (10), so we just calculate the elements required in ( + 1): By this way, the subspace matrices can be obtained through the above method; then the predictor will be calculated using (14) in open-loop system and (26) in closedloop system. So we can get the control input at instant + 1. At the next time, repeat the above procedure to implement the online adaptive mechanism and it will result in a quicker response to process changes.
In presence of noise and online disturbance, it would result in an inaccurate identification precision and an unneglectable match error as the presence of the harmful (or useless) data in the online excitation. In our previous work [26], an inspection strategy of model precession was proposed, but it cannot eliminate the negative impact of harmful (or useless) data on system performance. In this paper, a data inspection strategy introduced is the use of prediction error to remove the harmful (or useless) data.
Calculate the following prediction error before adding new data: where +1 is the process output at + 1 time and̂+ 1| is the predictive output at time predicting + 1 time before adding new data. Similarly, the prediction error after adding new data can be also introduced: wherê+ 1| is the output at time predicting + 1 time after adding new data. While +1| ≤ +1| , the new data is a harmful (or useless) one, so maintain matrix and the system model invariably. Inversely, while +1| > +1| , use the new data to update the matrix and predictor. At the next sampling time, when the new data arrives, recycle the above progress.
For the sake of clarity, the proposed adaptive mechanism implemented in the closed-loop data-driven predictive controller is summarized in Algorithm 1.

Simulation Examples
In this section, a SISO (single input single output) example and a MIMO (multiple input multiple output) example identified and controlled by the proposed method are presented and discussed as follows.
Remark 1. The data used were preprocessed with the methods in Section 14 of [35].

A Hair Dryer
Example. This hair dryer system is a simple mechanical device. The input is the power of the heating device, which is a mesh of resistor wires. The output is the outlet air temperature, which can be measured by thermocouple. Air is fanned through a tube and heated at the inlet. The details can be seen in [35]. In this example, we operated in case of closed-loop system. was chosen to be a binary random signal shifting between 35 W and 65 W. The length of samples and sampling time were set to 1000 and 0.2 s, respectively. Firstly, totally 100 samples were used to verify the identification accuracy. The comparisons in Figure 2 show the response of the identified model and process output using open-loop data-driven predictive control (ODPC) in Section 1 and closed-loop data-driven predictive control (CDPC) in Section 2, where "Rf" is process output, "open-loop" is open-loop identified model, and "closed-loop" is closed-loop identified model. To test the cross-validation in Figure 2, a form of prediction error in [10] is given as where and are the values at instant of process and model output, respectively. Table 1 illustrates the prediction errors of open-loop and closed-loop identified models.
The cross-validation results indicate that the closed-loop model is more accurate than open-loop model. Then, the system is given a performance of desired output changes to track using ODPC and CDPC. The sample was set to 1000 and the sampling time used was 0.2 s. The tuning parameters used in this simulation were = = 3, = 3 , and = 0.16 * 3 . Figure 3 depicts the output tracking performance. It can be seen that CDPC shows the favorable control performance and has a better tracking ability compared to ODPC.
In order to verify the adaptive mechanism in Section 4, the model of closed-loop identification was identified as a state-space model: (1) Construct the block Hankel matrices from the closed-loop data.
(3) Compute the closed-loop subspace matrices and using (25). (4) Derive the predictor̂of predictive controller with (26). (5) Implement the control input using (27) and (28). (6) At the next time, when new data arrives, implement the data inspection strategy. If the data is harmful (or useless), keep the constant. Otherwise, implement the following steps. (7) Build the new input-output Hankel matrix * and the new R matrix is the QR decomposition results of * with (35). (8) Recursively computer the elements 11 , 21 , 22 , 31 , 32 of R matrix using (38) and (39). (9) Calculate the new subspace matrices using (40) and computer the control input by repeating steps 4-5. Then, back to step 6. For comparison, the adaptive methods in [25,26] are given. The method in [25] is an original receding window method which is performed by only decomposition. In [26], recursive approach is presented to obtain every element value of matrix and the model inspection strategy is given. Figure 4 shows the response comparison in the presence of disturbance after the system model changes. We can get that, in performance of disturbance rejection, the method in this paper is better than the other two methods. The data inspection strategy makes the contribution for this result. The harmless (or useless) data are always produced when we implement online identification. The control performance depends on the better data preprocessing in this paper compared to the methods in [25,26].
By comparing computation time of 1000 samples, the methods in [25,26] and this paper take about 71 s, 62 s, and 52 s, respectively. The method in [25] requiring decomposition at every instant results in the most time of the three methods. The computation time of our proposed method is less than that taken by the method in [26] since it only requires calculating the key elements 11 , 21 , 22 , 31 , and 32 of matrix of our method but every element value of matrix of the method in [26].
Additionally, to verify the usefulness of the data inspection strategy, the prediction error in (43) is used. When system model was changed, we introduced two identification ways, the data inspection strategy is used in one way and the other not. The prediction errors of these two ways are showed in Table 2 from 600 s to 1000 s. We can get that the data inspection strategy improves the accuracy of the method.

An Industrial 4-Stage Evaporator
Example. The evaporator is a nonlinear and time-varying industrial process control system, and considering the stability of system the evaporator is often necessary to work in the closed-loop case. The conventional control methods, such as PID control, Abstract and Applied Analysis The method in [25] The method in [26] The method in this paper   will result in poor control performance. The product quality will be also affected accordingly. The evaporator is used to reduce the water content of a product and is widely applied in chemical industry, food industry, pharmaceuticals, and others. Therefore, it is of an extremely important practical significance to use an effective control method to achieve fast and accurate control performance of the evaporator. A typical industrial 4-stage evaporator system and the detailed principle of operation can be seen in [36]. The system has three inputs and three outputs. The three inputs are input product flow , vapour flow V to the first evaporator, and cooling water flow to condenser, respectively. The three outputs are dry matter content TDS of output product, output product flow , and output product temperature , respectively [37].
The open-loop and closed-loop identification algorithms are applied in system. Using 1000 validation data for identification, the prediction errors in (43) are given in Table 3.
It is similar to the hair dryer example in Section 5.1; the closed-loop identification computes a more accurate model.  The target is set for the output TDS tracking the reference signal in the system. The parameters of proposed data-driven adaptive predictive control (DAPC) method were tuned as = 4000, = 1s, = = 10, = 30 , and = 0.1 * 30 . The initial value of TDS was 1.5 mg/L. For comparison, the recursive adaptive subspace predictive control (RASPC) method in [23] and an adaptive fuzzy-PID controller in [38] were selected as competitors to compare the tracking capability. Figure 5 depicts the tracking comparison of these three controllers in the first 2000 samples and Figure 6 showed the partial enlarged drawing between 1000 s and 1200 s of Figure 5. At 1600 s, we changed to increase by 10 percent; the response comparison after the parameters change is showed in Figure 7.
Through the simulation results, it may fairly be said that our proposed method is much better in output tracking and disturbance rejection than that performed by recursive method in [23] and fuzzy-PID controller in [38]. It can be interpreted that the reduction of the influence of the old data plays an important role.
As for the computation time, our method takes about 76 s for 1000 samples, while their recursive method in [23] is about 64 s; the latter method is somewhat superior to ours because it needs to add new data and eliminate old data at every instant in ours but theirs only add new data.
Similar to Section 5.1, the prediction errors with and without the data inspection strategy from 1600 s to 2000 s are listed in Table 4. Corresponding to the conclusion in the hair   dryer example, the superior performance is obtained as using the data inspection strategy.

Conclusion
In this paper, the design of a data-driven adaptive predictive controller based on closed-loop subspace identification has been addressed. The predictor is identified through the closed-loop subspace identification and used to design a data-driven predictive controller. The adaptive mechanism is presented that combines the merits of both receding window and recursive identification methods, keeping the size of input-output data matrix constant and using recursive identification to obtain the subspace matrices which can derive the predictor. Meanwhile, the data inspection strategy is used to eliminate the new harmless (or useless) data. By simulation studies for two examples its performance has been proved to be efficient by comparing with other methods.