Desired Terminal State Concept in Model Predictive Control: A Case Study

The paper deals with an online optimization control method for dynamical processes called Model Predictive Control (MPC). It is a popular control method in industry and frequently treated in academic areas as well. The standard predictive controllers usually do not guarantee stability especially for the case of short horizons and large control error penalization. Terminal state is one way to ensure stability or at least increase the controller robustness. In the paper, deviation of the predicted terminal state from the desired terminal state is considered as one term of the cost function. Effect of the stability and control quality is demonstrated in the simulated experiments. The application area for online optimization methods is very broad including various logistics and transport problems. If the dynamics of the controlled processes cannot be neglected, the optimization problemmust be solved not only for steady state but also for transient behaviour, e.g., by MPC.


Introduction
Minimization of a quadratic cost function is a common method for solving many engineering problems. In the control area, this method is fundamental not only for standard control design methods like optimal control, e.g., [1,2] but also for a state estimation [3]. For example, well known Kalman estimator [4] was published in 1960. Current state of HW and SW technology allows us to look back a bit, modify, and apply some methods well known from the past but not used practically. In the contrary to standard PID controllers such methods have potential to increase control quality and solve more complicated and complex tasks. Usability, reliability, robustness, and of course also the price of such a system is the other side of the coin. In the paper we are introducing MPC desired terminal state calculated from steady state and we test it by simulation for higher order single-input single-output process.
The paper is structured as follows: standard controller design is described in Section 2, modified method is introduced in Section 3, simulated control experiments are presented in Section 4, and conclusions are given in Section 5.

Standard Controller Design
Under the assumptions of linear controlled system and quadratic cost function it is possible to formulate the task of the optimal controller design as a standard mathematical problem, extreme finding with an analytic solution. A unique solution exists also in the case of constrains existence in a form of linear inequalities.
The key part of the controller design is to incorporate maximum of the known information and demands into the properly formulated cost function. It is possible to involve various (even conflicting) control demands. Then the controller tuning consists in weightings of the particular demands.
From practical point of view, it is appropriate to formulate the task in discrete-time domain with receding (finite) control horizon [5][6][7][8]. The length N of the horizon is a parameter in the control design. The general formulation of a set-point tracking task is given by (1a), a state space description of the controlled dynamical linear system with state and input variables constraints, and by (1b), a quadratic cost function J (control objective) with three terms. The cost function J depends on the horizon length N, the initial state x(k) (initial 2 Journal of Control Science and Engineering conditions in time k), and the time course of the future setpoint w N (vector along the control horizon). The solution consists in computation of such a vector of system inputs u N , which leads to the minimum of the cost function and simultaneously respects all constrains.
x ( + 1) = Ax ( ) + Bu ( ) where A, B, C, D are parameters of a discrete-time dynamical process model and H, h, G, g are parameters of state and input variables constraints.
where Q x , Q, and R are weighting matrices of particular terms. The cost function always contains the fundamental control requirement, the term : the controlled outputs y of the system should follow the set-points w. This basic requirement is usually followed by another term of the cost function. The term implies the control costs: the set-point tracking is desired but not at the cost of arbitrarily large control actions. The term J x in the cost function can be used only in the case of finite control horizon and state space description. It introduces into the cost function a dependence on the system state at the end of the control horizon called terminal state. The predictive controller design based on input-output description does not use it in a basic formulation of the cost function. The terminal state is obviously introduced in the extensions concerning the stability and robustness; see, e.g., [9][10][11][12][13][14][15]. The terminal state brings into the cost function dependence on all state variables. The standard cost function depends only on the system outputs (or control error) and it can be independent from some state variables-this is given by matrix C. Thus some states can increase ad infinitum even if the cost function is finite. In the case of control design based on input-output models, where state does not exist in a nature form, the terminal state is replaced with a sequence of input and output variables. That approach of the terminal state is called in the literature "terminal constrains" [16][17][18][19][20].
In some cases the terminal state is important from the mathematical point of view. In case of LQ control design on finite horizon, the mathematical importance of the terminal state is that the matrix Q x determines the initial value of a working matrix which is developing by iterating solution of the discrete Riccati equation.
In literature the terminal state is obviously mentioned only in the context of the controller stability. The use of the terminal state has also an implication to the controller performance. The standard formulation of the terminal state in the form of (1b) leads to the permanent steady state control error in case of nonzero set-point. This problem can be easily solved by the terminal state in a form of the deviation from a the desired terminal state x w . The desired terminal state is a function of the set-point and/or other demands. Additional optimization in steady state can be an integral part of the controller due to the desired terminal state concept. Under the "optimization in steady state" we understand that controller ensures minimum of the weighted quadratic norm of a vector of deviations between desired and calculated terminal state.
Clear and unique additional requirements can be formulated because the state vector contains complete information about the state of the system. The predictive controller can ensure, e.g., demand of minimum energy cost of a system with more inputs than outputs (nonsquare, overactuated system). The problem of how to determine an optimal steady state for such systems is discussed, e.g., in [21].
Application area of predictive control methods is not limited to refinery, chemical, pulp, and paper industries, but it is becoming very broad. It can be advantageously applied also in transport industry, as demonstrated in [22] for traffic signal control based on traffic density prediction or in [23], where the authors propose the MPC algorithm for automatic train operation system.

Modified Controller Design
The controller design starts from a discrete-time state space model of the controlled MIMO (Multi-Input Multi-Output) system with n u inputs, n x state variables, and n y outputs. The model is in a standard form (3a); we suppose matrix D = 0. terminal state x(k+N+1) depend on the actual state x(k) and on a vector of future inputs u N1 .
x ( + + 1) = S x ( ) + S u 1 y = S x ( ) + S u 1 (3b) Matrices S xx , S xu , S yu a S yu depend on the state space model parameters according to (4c).
With respect to a terminal state in the cost function (5) in time instant k+N+1, the input vector has to be of length k+N and thus the vector is marked as u N1 . On the other hand the last item in the vector u N1 does not influence output vector y N . Because of this the last column of the matrix S yu (4c) is filled with zeros.

CA
The cost function in matrix form (5) changes from (1b) because of the terminal state application as a deviation from the desired terminal state x w and the vector of manipulated variable ûu N1 is calculated as a deviation from the supposed future inputs u 0,N1 .
( , u 1 , x ( ) , w ) = e Qe + Δu 1 RΔu 1 + +Δx ( + + 1) Q Δx ( + + 1) where N is length of control horizon, x w is desired terminal state, First item of the vector u N1 is applied as a control action u(k) every time instant and whole procedure is repeated. Constant vector filled with values of u(k-1) is used as supposed future inputs (vector u 0,N1 ) in the following simulations. Another possibility how to choose the supposed future input vector u 0,N1 is to use shifted vector u N1 from the previous calculation step. Both approaches are identical in principle, but the control response differs because of the effect of changed weighting proportions.

Desired Terminal State.
Computation of the desired terminal state is trivial in case of the system with identical number of inputs and outputs and if we consider steady state. The controlled system steady state behaviour is given by The solution for the desired output y 0 =w 0 is

Simulated Experiments
The aim of the following control simulations is to demonstrate the effect of the terminal state in predictive controller design to the control quality and stability. The simulations are supposed as an ideal case; controlled system is identical with the process model used for the controller design and neither noises nor disturbances are considered. The controller is designed for the set-point tracking task. Two different controlled systems are treated in the simulations. The first system is a standard system of a higher order (8a) and the second one is a system with nonminimum   Figure 1 ( ) = (9 + 1) (9 + 1) (5 + 1) 5 (8a) ( ) = (9 + 1) (−9 + 1) (5 + 1) 5 (8b) Standard predictive controller described in Section 2 operates without any modifications with MIMO or SISO systems. The sampling time and the control period is T = 1 s. The weighting matrices Q x , Q, and R are diagonal. All diagonal elements of the weighting matrices are constant and their values are chosen so that the weight of every term in the cost function is comparable. From this reason the weighting matrices are computed as reciprocal quadratic norms of the corresponding steady state vectors according to (9). Tuning parameters of the controller are relative weightings and x .
where x 0 is steady state vector, y 0N is vector of constant outputs, u 0N is vector of constant inputs, x is relative weight of matrix Q x , is relative weight of matrix R and I is identity matrix of appropriate dimensions.
The set-point shape consists from tree parts. The first part takes the same time as a control horizon plus 5 sampling periods and the set-point is constant. The second part lasts as a system settling time (50 s) and the set-point linearly increases from the first to third part. The third part is as long as the second one and the set-point is constant again. The control quality measure is calculated as an integral of the absolute control error where NS is number of samples during the control experiment.
The effect of the terminal state can be observed from IAE measure values for simulated control experiments summarized in Table 1 for system (8a) and in Table 2 for system (8b).   The control experiments were simulated for several values of control horizon length N and terminal state relative weights .
The control responses of two selected control experiments are plotted in Figure 2(a), controlled system (8a), and in Figure 2(b), controlled system (8b). Both experiments are considered with same parameters: the length of the control horizon is N = 35 and the relative gain of the terminal state is

Conclusion
Effect of the terminal state to the stability of the control is definitely positive. Even in the case of wrong choice of the controller parameters (control horizon is too short) the terminal state increases dramatically the controller stability.
Nevertheless the control quality is obviously worse if the terminal state is used. The control deterioration is evident even if the terminal state was considered in a form of the deviation from the desired terminal state calculated from the steady state. The deviation form solves the main problem, permanent control error in the steady state. Control quality decrease is caused by stronger effect of the terminal state in the cost function then the other two terms (although in one point of the control horizon). This statement is not true generally because the effect depends on the controlled system and on the controller parameters, firs of all on the length of the control horizon. This effect was strong especially by the control of system (8a). The cost function minimization leads to large initial items of the calculated vector ûu N1 even if only one item of the set-point vector w N is changed at the end of the horizon, the last item of the vector w(k+N+1). This situation is demonstrated in Figure 3. The relative weight of the terminal state deviation is x = 1.0. The diagonal elements of matrix Q x = 1.6e-8, diagonal elements of matrix Q = 2.94e-2, and diagonal elements of matrix R = 2.85e-1.
The desired terminal state in our case is a steady state corresponding to the set-point at the end of the control horizon. The question remains whether to use more accurate desired terminal state. This should increase the control quality, but it will make the whole controller more complicated and we will lose interesting feature for the steady state optimization especially for nonsquare systems.

Data Availability
The process models and controller parameters data used to support the findings of this study are included within the article. We can provide also MATLAB scripts with the code for the simulations if suitable.

Conflicts of Interest
The authors declare that they have no conflicts of interest.