Comprehensive Control of Networked Control Systems with Multistep Delay

In networked control systems with multi-step delay, long time-delay causes vacant sampling and controller design difficulty. In order to solve the above problems, comprehensive control methods are proposed in this paper. Time-delay compensation control and linear-quadratic-Guassian (LQG) optimal control are adopted and the systems switch different controllers between two different states. LQG optimal controller is used with probability 1 − α in normal state, which is shown to render the systems mean square exponentially stable. Time-delay compensation controller is used with probability α in abnormal state to compensate vacant sampling and long time-delay. In addition, a buffer window is established at the actuator of the systems to store some history control inputs which are used to estimate the control state of present sampling period under the vacant sampling cases. The comprehensive control methods simplify control design which is easier to be implemented in engineering. The performance of the systems is also improved. Simulation results verify the validity of the proposed theory.


Introduction
Feedback control systems in which control loops are closed through a real-time network are called networked control systems (NCSs) [1,2]. This type of systems has many advantages such as flexible system design, simple installation and maintenance, increased system agility, and reduced system wiring. Compared with conventional point-to-point control systems, however, the insertion of communication network in the feedback control loop makes the analysis and design of NCSs more complex. Conventional control theory with many ideal assumptions, such as synchronized control and nondelayed sensing and actuation, has to be reevaluated before it is applied to NCSs.
The existence of network in NCSs inevitably induces some nondeterministic phenomena, especially networkinduced delay, which will degrade the performance of control systems and even destabilize the systems. The networkinduced delay mainly comes from two resources: sensor-tocontroller delay and controller-to-actuator delay. It is of great significance to explore suitable control methods under the stochastic delay in NCSs to improve the system performance.
Some researchers studied stochastic time-delay and the optimal controllers of NCSs whose network-induced delay was shorter than a sampling period [3][4][5]. Nevertheless, the network-induced delay is longer than one sampling period in many cases.
Substantial efforts have been done for the nonlinear NCSs with time-varying delay. A time-based neuron-dynamicprogramming (NDP) optimal control scheme for uncertain nonlinear NCSs was introduced by using output feedback and without utilizing value and policy iterations [6]. Closed-loop stability in the mean was demonstrated by selecting novel neural-network (NN) update laws. However, the stochastic characteristic of time-delay was not referred in the optimal control design. Cao [7] provided improved time-delaydependent stability criteria for multi-input and multi-output (MIMO) NCSs with nonlinear perturbations. Related control methods and simulations were not given to support the proposed theory. A high frequency NCS was described by 2 The Scientific World Journal a time-varying delayed delta operator system with a high frequency constraint [8]. An improved stability condition was given for the delta operator system by using a generalized Kalman-Yakubovic-Popov lemma. They did not discuss the stochastic characteristic of time-delay and concrete control methods. In [9], optimizing controller design of real-time NCSs was presented based on different models. For timedelay less than one sampling interval, they modeled the system as a time-invariant control system with constant timedelay. For time-delay greater than one sampling interval, they modeled it as a jump linear control system. Nevertheless, NCSs models should be nonlinear because of time-varying delay. Using the Lyapunov-Krasovskii method, a sufficient condition for asymptotic stability of nonlinear NCSs was provided in [10]. Measured values were asynchronously sampled and transmitted over multiple communications links. The effects of communication in each link were captured by a time-varying delay element. In order to avoid complexities of these kinds of nonlinear NCSs, an assumption of delay bound was made under which the NCSs models were not sensitive to asynchrony of sampling and transmission between different links.
With regard to the characteristic of uncertain delay in NCSs, some researchers proposed predictive control methods to compensate time-delay whose probability distribution was unknown [11][12][13][14]. The stochastic time-delay system was transformed to a deterministic delay system by placing a special amount of buffers at the nodes in NCSs. Multistep predicting controllers were designed to improve system performance through model matching and multistep predictive output compensation. However, this kind of models is too complex and has high computational cost due to the uncertainty of time-varying delay.
When network-induced delay has a known probability distribution and is longer than one sampling period, Yu et al. [15] proposed a control mode: sensor and actuator were time-driven and controller was event-driven. Ma and Fang [16] put forward a time-division control mode: sensor was time-driven, controller was event-driven, and actuator was time-division-driven. Under the two kinds of control modes, perhaps there were not new control inputs arriving at the actuator during a sampling period because of data congestion or network bandwidth limitation. In this case the old control input of last sampling period continued acting on the plant, which might induce long time-delay. This kind of situation is called vacant sampling. The vacant sampling in NCSs may degrade the performance of control systems and even destabilize the systems. Furthermore, it will make optimal controller design difficult to be implemented in engineering when the time-delay of NCSs is too long. It is important to take some measures for compensating vacant sampling and long time-delay.
To solve all the aforementioned problems, comprehensive control methods of NCSs are proposed in this paper. Timedelay compensation control and linear-quadratic-Guassian (LQG) optimal control are adopted and the systems switch different controllers between two different states. LQG optimal controller is used with probability 1 − in normal state, which is shown to render the systems mean square exponentially stable and guarantees the performance of NCSs with big probability. Combined with the strength of predictive control methods, time-delay compensation controller is used with probability in abnormal state with long delay and compensates vacant sampling and long time-delay. The stochastic characteristic of time-delay is also considered to design the optimal control of NCSs. The proposed comprehensive control methods reduce the complexities of system control and computational cost under the guarantee of improving system performance.
The rest of this paper is organized as follows. Section 2 describes comprehensive control methods of NCSs. Simulations are given in Section 3, followed by conclusions in Section 4.

Description of Comprehensive Control Methods
The stochastic delay in NCSs mainly comes from two resources: sensor-to-controller delay sc and controller-toactuator delay ca . Assume that network-induced delay has a known probability distribution function and = sc + ca ≤ ( is a positive integer, ≥ 2, and is the sampling period of sensor). Now, we describe the model of NCSs. We assume the state equation of plant is linear time-invariant which is expressed as where ( ) ∈ , ( ) ∈ . , , and are matrices of appropriate sizes. Discretizing (1) separately under Yu's control mode [15] and Ma's control mode [16] over a sampling interval [ , ( + 1) ), we have a stochastic NCSs model as follows: where under Yu's control mode: and under Ma's control mode: The Scientific World Journal Assume that the transfer delay from sensor to actuator at the moment is (0 ⩽ ⩽ ). We can conclude that { , = 0, 1, 2, . . .} is a Markov chain. The state transition matrixes under Yu's control mode and Ma's control mode can be derived [15,16].
Under the two kinds of control modes above, perhaps there is vacant sampling during a sampling period because of data congestion or network bandwidth limitation. For the sake of avoiding the vacant sampling in NCSs, time-delay compensation control is put forward and used to compensate long time-delay in this paper. It is described as follows.
A buffer window is established at actuator to store some history control inputs { − , − +1 , . . . , −1 }. There are at most history control inputs in the window. If vacant sampling occurs in NCSs over a sampling interval [ , ( + 1) ), the history control inputs are used to estimate the control state of present sampling period. According to the time order of history control inputs, some weights are derived. The average value of history control inputs with weights is calculated to act on plant instead of the old control input of last sampling period. In terms of the synthetic effects of these history control inputs, the performance of NCSs is improved. At the beginning, the initial values of control inputs in the buffer window are set as 0. We use the following equation to estimate control input which is denoted as over a sampling interval [ , ( + 1) ). The estimation value of is denoted bŷ= where [( − 1)/2] is the integer function of ( − 1)/2 and [ /2] is the integer function of /2. The time-delay compensation control guarantees not only the priority of history control inputs with smaller delay but also the synthetic effects of all history control inputs. In NCSs, optimal control is often used to improve system performance and stabilize the whole system. Now we design the LQG optimal controller of system (2). Theorem 1. With the system having full state information, the LQG optimal control law for NCSs system (2) is where and it renders system (2) mean square exponentially stable.
Proof. In this paper, we design a control law of the stochastic open-loop NCSs as expressed in (2) to minimize the cost function Then system (2) can be expressed as follows: ] . 4 The Scientific World Journal Cost function (10) is equivalent to ) .
Equation (16) is Bellman functional equation. Then, we prove that the solution of (16) is as follows: where and are nondeterministic. We prove (17) with mathematical induction. Let express time. When = , the conclusion is apparently correct. If we suppose that when = + 1 the conclusion is correct, we will prove that when = the conclusion is also correct. Consider (b) When = + 1, (17) holds. Now, we have Using (12) we can conclude The Scientific World Journal 5 (c) = Using (20) and considering (16), we can derive ( , ) is minimum cost, so is . Similar to the proving process in [17], we can conclude that LQG optimal control law (8) renders system (2) mean square exponentially stable. Now we prove Theorem 1.
In practical application of NCSs, network-induced delay is often longer than one sampling period. When the timedelay of NCSs is too long, it will make optimal controller design difficult to be implemented in engineering. In order to reduce the complexities and computational cost of system control, comprehensive control methods are proposed whose main idea is as follows.
When time-delay is smaller than a suitable delay bound, optimal controller (8) is used to stabilize the systems during most of running time, which can make optimal control easier to be implemented. While time-delay is bigger than the delay bound, time-delay compensation controller (5) is used to compensate vacant sampling and long time-delay.
The comprehensive control methods are based on confidence level in this paper. Normal and abnormal states are defined for the delay's two different cases. The related definitions are given as follows.
Definition 2 (see [18]). Assume that the distribution function of population is ( ; ), is an unknown parameter, and ∈ Θ. For a constant (0 < < 1), if statistic variable = ( 1 , 2 , . . . , ), which is derived from the samples 1 , 2 , . . . , , meets the following equation: then stochastic interval (−∞ ) is called single-side confidence interval with confidence level 1 − . that is called single-side confidence upper limit. Now we discuss the single-side confidence interval of time-delay based on confidence level in NCSs. For given constant (0 < < 1), the single-side confidence upper limit = ( is a positive integer and < ) of can be derived. On the basis of and , normal and abnormal states are defined as follows.
In NCSs, time-delay does not always access or reach the maximum delay . In general cases, it is near expectation value { } with big probability. We can choose suitable confidence level to make random event { < } be a bigprobability event when random event { ≤ ≤ } is a small-probability event. The probability of the systems in normal state is 1 − . LQG optimal controller (8) is adopted in normal state, which is shown to render the systems mean square exponentially stable during most of running time. The probability of the systems in abnormal state is . Time-delay compensation controller (5) is adopted in abnormal state to compensate vacant sampling and long time-delay.
Using Schur complement and Cone-complement linear technique, an approximate solution of control law was obtained in [19], which rendered system (2) asymptotically stable. It is described as follows.
When the maximum delay is known, the controller = can be obtained based on Theorem 4 using the MATLAB LMI Toolbox. Now we make some changes in the comprehensive control methods. LQG optimal controller (8) is adopted in normal state, which is shown to render the systems mean square exponentially stable. Controller (28) is adopted in abnormal state to compensate vacant sampling and long time-delay, which is shown to approximately render the systems asymptotically stable. Then the system model is described as follows.
We denote the state variable and output of NCSs in normal state and abnormal state as 0 , 0 , , and . Then the optimal controller in Theorem 1 can be written as where 0 ∈ 1× and ∈ 1× ( = 1, . . . , ). Time-delay compensating controller (28) can be written as From the discussion above, we get the comprehensive control model of system (2)

Simulations
The simplified model of the inverted pendulum process is as follows [20]: In this paper, we use MatLab and C++ to simulate the comprehensive control methods on the networked inverted pendulum system that we construct based on NS2 [21]. In the simulations, parameters are selected as follows: = 0.05 s, = = [ 1 0 0 1 ], and = 0.1. It is assumed that the maximum network-induced delay is 3 ; that is, = 3. Using stochastic sampling experiment, we can get = 2 ( = 2) under confidence level parameter = 0.05. The state transition matrix under Yu's control mode is From Figures 1 and 2, we can see that the system performance is obviously improved under the comprehensive control methods which make NCSs faster to reach stability status. The simulation results show the validity of the proposed theory.

Conclusions
In order to improve the performance of NCSs with multistep delay, comprehensive control methods based on confidence level are presented in this paper.
Time-delay compensation control and LQG optimal control are adopted and the systems switch different controllers between two different states. LQG optimal control is used with probability 1 − in normal state, which is shown to render the systems mean square exponentially stable. Timedelay compensation control is used with probability in abnormal state. The comprehensive control methods simplify controller design and reduce computational cost. It is proved by simulations that the new control methods have better control effects than single control method. We will synthesize more optimal control methods in future work to further improve the performance of NCSs.