Probability-Weighted Optimal Control for Nonlinear Stochastic Vibrating Systems with Random Time Delay

A probability-weighted optimal control strategy for nonlinear stochastic vibrating systems with random time delay is proposed. First, by modeling the random delay as a finite state Markov process, the optimal control problem is converted into the one of Markov jump systems with finite mode. )en, upon limiting averaging principle, the optimal control force is approximately expressed as probability-weighted summation of the control force associated with different modes of the system. )en, by using the stochastic averaging method and the dynamical programming principle, the control force for each mode can be readily obtained. To illustrate the effectiveness of the proposed control, the stochastic optimal control of a two degree-of-freedom nonlinear stochastic system with random time delay is worked out as an example.


Introduction
In many complicated control systems, such as manufacturing plants, vehicles, aircraft, and spacecraft, a communication network is used to gather sensor data and send control signals.Time delay occurs while exchanging data among these devices, which is usually random [1][2][3].Time delay, particularly random time delay, can dramatically degrade the performance of the control system and even destabilize it.erefore, time delay should be taken into account when designing control strategy.
e control problem of linear or nonlinear systems with constant time delay has been examined in the literatures [4][5][6][7].Far less known is about the systems with time-varying delay [8,9], especially random time delay.A key difficulty is how to model the random time delay.A simple method was to regard the delay time as a constant [10].However, the delay time is much longer than necessary, which usually results in poor performance.Randomness of the delay time was taken into account by Nilsson et al. [11].To keep the model simple for analyzing, the current delay is assumed to be independent of previous delay times.However, in real system, the current delay is usually correlated with the last one.us, a more reasonable way is to model the random delay time as a Markov jump process [12].Based on this model, a finite horizon optimal control of linear systems with randomly varying time delay is studied [13].Zhang et al. [14] investigated feedback stabilization of randomtime-delayed linear system.Wu et al. [15] analyzed the stability of the stochastic linear system with time-varying delay.Wang et al. [16] investigated the problem of robust fault detection for networked Markov jump systems with random time delay.e problem of robust H∞ control for a class of uncertain stochastic system with random delay against actuator failures is studied by Sakthivel et al. [17].Huan et al. [18] investigated the dynamics of nonlinear stochastic systems with random time delay.e previous works focused mainly on the linear systems.Far less known is about the nonlinear systems.
In this paper, we present a probability-weighted optimal control strategy for nonlinear stochastic systems with random time delay.e organization of this paper is as follows: in Section 2, such a system is converted into the Markovian jump one by modeling the random delay as a Markov process.Section 3 converted the system into the one with time delay as a parameter.Probability-weighted optimal control law is determined in Section 4. In Section 5, the application and effectiveness of the proposed procedure are demonstrated by an example.A summary of findings is given in Section 6.

Formulation of Problem
2.1.Modeling of Random Delay.For many complex systems, i.e., networked control systems, the random delay time τ(t) usually has finite number of levels due to different network traffic conditions [11].τ(t) may switch randomly from one level to another with time.Usually the switching of different delay levels follows the rules of Markov chain.en, the random time delay can be modeled as the following Markov jump form: where s(t) is a finite state Markov process, which points to the mode of time delay.s(t) takes discrete values from a finite set S � 1, 2, . . ., l { } with the following mode transition probability [18,19] where Δt > 0 is a sufficiently small time interval; lim Δt ⟶ 0 (o(Δt)/Δt) � 0; P s(t + Δt) � j|s(t) � i   denotes the transition probability from mode i to mode j; λ ij ≥ 0 is the transition rate; and λ ii � − l j�1 j≠i λ ij .

Nonlinear Stochastic System with Random Delay.
Consider a multi-degree-of-freedom (MDOF) nonlinear stochastic system with random time delay.e motion equation of the nonlinear stochastic system is of the following form: where g i (X i ) is the nonlinear restoring force; ε is a small parameter; f ij (X, _ X) denotes coupled nonlinear damping; h ik (X, _ X) represent the coefficients of random excitations; and ξ k (t) are Gaussian white noises in the sense of Stratonovich with zero mean and correlation functions ] are feedback control forces with random delay time.By using Equation (1), system (3) can be rewritten as the following Markov jump form: Note that by modeling the random delay time as a finite state Markov process, system (3) has been formulated as a Markov jump system with finite mode.e optimal feedback control problem of system (3) then can be studied in the framework of the Markov jump system.
e purpose of the present study is to derive an optimal feedback control law to minimize the response of system (3) or (4).For semiinfinite time interval control, the performance index can be expressed as follows: where L is called cost function and

Converted System with Time Delay as a Parameter
System (4) has the following randomly periodic solutions [5]: where V i denotes the instantaneous frequency and Φ i , A i , Θ i , V i , and Γ i are all random processes.For the case of small time delay, i.e., 0 ≤ τ (u) ≪ 1 (u ∈ S), it is proved that the following approximate expressions hold [5]: where ω i � ω i (A i ) is the averaging frequency which is the average of the instantaneous frequency V i , i.e., and 〈〉 Φ i represents the averaging with respect to According to expression (7), the time-delayed control forces can be approximately expressed in terms of the state variables with time delay as a parameter, i.e., ). en, Equation (4) becomes 2 Shock and Vibration

Probability-Weighted Optimal Control Law
According to limiting averaging principle [20,21], the solution of Equation ( 9) converges in probability to the solution of the following probability-weighted averaged equations as ε ⟶ 0: where T � [τ (1) , τ (2) , • • • , τ (l) ] T and U i (X, _ X; T) is probability-weighted control force, given by where ) is the control force when s(t) is fixed at s(t) � u and μ (u) is the stationary probability distribution of s(t) for s(t) � u, which satisfies [18].
where λ ui is transition rate given in Equation (2).Combining the normalizing condition  l u�1 μ (u) � 1, the stationary probability distribution μ (u) can be calculated.
Since probability-weighted averaged equation ( 10) is an approximation of the original system (3), the optimal control force U * i in Equation ( 11) is near optimal for the original control problem.
Own to the relationship in Equation ( 11), the optimal control force  U ′ * i (X, _ X; τ (u) ) for a fixed mode should be determined first.Let the Markov jump process be arbitrarily fixed at s(t) � u, Equation ( 9) can be rewritten in the fixmode quasi-Hamiltonian form: where X i � Q i , _ X i � P i .e Hamiltonian system associated with Equation ( 13) is fully integrable.e Hamiltonian H is where ) dv i are called first integrals.According to the stochastic averaging method for quasiintegrable Hamiltonian systems [22], the partially averaged Itô equations associated with system (13) are obtained: where T and m r (H) and σ rk (H) are the averaged drifts and diffusion coefficients respectively, given by Accordingly, the performance index in Equation ( 5) becomes Based on the stochastic dynamical programming principle, the following dynamical programming equation (DPE) can be established for averaged system (15) and performance index (17): where )ds is optimal average cost.e optimal control force  U ′ * i then can be obtained by minimizing the right-hand side of Equation (18), i.e.,

Shock and Vibration
where L 1 (H) ≥ 0 and R is a positive-definite symmetric matrix.Substituting Equation (20) into Equation ( 19) and exchanging the order of the deriving with respect to  U ′ i and averaging, the optimal control force for a fixed mode of system (13) can be obtained: where V is undetermined.By inserting  U ′ * i into Equation ( 18) for replacing  U ′ i , the final DPE can be obtained.V will be then determined by solving this equation.
Note that the optimal control force in Equation ( 21) is a function of current system's states P j (t).However, in system (4), the delayed states are observed.us, the optimal control force  U ′ * i should be expressed in functions of delayed states P j (t − τ (u) ) by using approximation relationship in Equation ( 7): Substituting Equation ( 22) into Equation ( 11), the optimal control force of the original system (4) is then obtained: It is seen that the optimal control force U * i is probabilityweighted summation of the control force associated with different modes of the system, weighted by the stationary probabilities μ (u) .e obtained optimal control force is continuous and independent of the Markov jump parameter, which can be easily executed by actuators.
Inserting U * i in Equation (23) into system (15) for replacing  U ′ i , the optimal controlled system is obtained.e response of the optimal controlled system will be predicted by solving the associated Fokker-Planck-Kolmogorov (FPK) equation or by using Monte Carlo simulation directly from the original system (4).e control effectiveness and control efficiency can then be evaluated (see the following example, i.e., Equations (32) and (33)).

Numerical Example
Consider two oscillators coupled by linear and polynomial type nonlinear dampings subject to external random excitations and random time-delayed feedback control.e motion equations of the system are of the following form: where W i (t) (i � 1, 2) are uncorrelated Gaussian white noises with intensities 2D i and U iτ are control forces with random time delay.ree-level time delay is considered here, which means s(t) takes values in set S � 1, 2, 3 { }: τ (1) for low network load, τ (2) for medium network load, and τ (3) for high network load [10].
Let X i � Q i , _ X i � P i .By using the approximation in Equation ( 7) and applying the stochastic averaging method, the partially averaged Itô stochastic differential equations for a fixed mode can be obtained as the form of Equation ( 15): where Based on the dynamical programming principle, the DPE is derived as the form of Equation ( 18), from which the optimal control force for a fixed mode can be obtained as the form of Equation ( 21), i.e., To obtain the value of function zV/zH r , the value function V is assumed to have the following polynomial form: L 1 (H) in Equation ( 20) is specified by Inserting Equations ( 27)-( 29) into the DPE (18), the coefficients of value function V can be determined.
Substituting Equation (27) into Equation ( 23), the probability-weighted optimal control force of system (24) is then obtained To evaluate the proposed control strategy, the control effectiveness and control efficiency are proposed: ey are the relative reduction in root-mean-square displacement and the ratio of the control effectiveness to the normalized root-mean-square control force, respectively.Obviously, higher K i and μ i values indicate a better control strategy.Some numerical results have been calculated as shown in Figures 1-4 for system's parameters: α 11 � 0.1, α 12 � 0.0, and D 2 � 0.05.e three-level time delay is specified by (34) ree special Markov jump rules are considered, with Observe that so that the system is more likely to take the low time delay mode s(t) � 1. e system favors the medium time delay mode s(t) � 2 if Λ � Λ 2 , while it favors the large time delay mode e stationary joint probability densities p u (Q 1 , P 1 ) of the first oscillator of the uncontrolled system and p c (Q 1 , P 1 ) of the optimal controlled system are, respectively, shown in Figures 1(a) and 1(c).Obviously, p c (Q 1 , P 1 ) has much larger mode and smaller dispersion around their equilibrium for optimally controlled system than those for the uncontrolled system. is implies that the proposed control law has high effectiveness for attenuating the system's response.
e results of direct Monte Carlo simulation of system (24) are also obtained and shown in Figures 1(b) and 1(d).Favorable agreements between these two results can be seen, which demonstrates the validity of the proposed method.
Figure 2 shows the stationary probability density p(Q 1 ) of displacement of the first oscillator of the optimal controlled system for different transition rules.It can be seen that p(Q 1 ) has the largest mode around and the mode will decrease as the system cycles through Λ � Λ 1 , Λ � Λ 2 , and Λ � Λ 3 .is implies that the higher probability of the system working in large time delay mode will lead to worse performance of the control force.In Figure 2, the lines are obtained by the proposed method, while the dots are obtained by direct simulation of system (24).Observe that the dots match closely with the corresponding lines.e same observation can be made about p(Q 2 ) in Figure 3.
In Figure 4, the variation of the control effectiveness K 1 and control efficiency μ 1 with excitation intensity 2D 1 is displayed for different transition rules.It is seen that the Shock and Vibration proposed control strategy keeps high control effectiveness and control efficiency for varying excitation intensities.It is also seen from Figure 3 that as Λ takes values through Λ � Λ 1 , Λ � Λ 2 , and Λ � Λ 3 and K 1 and μ 1 decrease monotonously.e observation can be explained in a similar fashion as in Figure 2. Figure 5 shows the control effectiveness K 2 and control efficiency μ 2 for varying excitation intensity 2D 2 . - Figure 2: e stationary probability density p(Q 1 ) of displacement of the first oscillator of optimal controlled system compared with uncontrolled system.e lines are obtained by proposed method, while the dots are obtained from Monte Carlo simulation of the original system (24).
- e lines are obtained by proposed method, while the dots are obtained from Monte Carlo simulation of the original system (24).
Figure 1: e stationary joint probability density p u (Q 1 , P 1 ) of the first oscillator of uncontrolled system and p c (Q 1 , P 1 ) of optimal controlled system: (a, c) results by the proposed method; (b, d) results by Monte Carlo simulation of the original system (24). 6

Shock and Vibration
Finally, sample time histories of the displacement Q 1 for the controlled system compared with uncontrolled system are displayed in Figure 6.

Conclusions
In this paper, a probability-weighted optimal control strategy for nonlinear systems with random time delay is proposed.Based on the random switch model of random time delay, the optimal control problem of such a system was converted into the one in framework of Markov jump systems.Upon limiting averaging principle, the optimal control force was approximately expressed as probabilityweighted summation of the control force associated with different modes of the system.e proposed optimal control force is continuous and independent of the Markov jump parameter, which can be easily executed by actuators.e feasibility and effectiveness of the proposed optimal control

Shock and Vibration 7
strategy were demonstrated by dealing with an example of a two-DOF nonlinear stochastic system with random time delay.

Figure 3 :
Figure3: e stationary probability density p(Q 2 ) of displacement of the second oscillator of optimal controlled system compared with uncontrolled system.e lines are obtained by proposed method, while the dots are obtained from Monte Carlo simulation of the original system (24).

Figure 4 :Figure 5 :
Figure 4: e control effectiveness K 1 and control efficiency μ 1 versus stochastic excitations intensity 2D 1 for different transition rules.

Figure 6 :
Figure 6: e sample time history of displacement Q 1 for controlled system compared with uncontrolled system.