Optimal Vibration Control of a Class of Nonlinear Stochastic Systems with Markovian Jump

The semi-infinite time optimal control for a class of stochastically excited Markovian jump nonlinear system is investigated. Using stochastic averaging, each form of the system is reduced to a one-dimensional partially averaged Itô equation of total energy. A finite set of coupled dynamical programming equations is then set up based on the stochastic dynamical programming principle and Markovian jump rules, from which the optimal control force is obtained. The stationary response of the optimally controlled system is predicted by solving the Fokker-Planck-Kolmogorov (FPK) equation associated with the fully averaged Itô equation. Two examples are worked out in detail to illustrate the application and effectiveness of the proposed control strategy.


Introduction
Markovian jump system (MJS) is a class of hybrid systems, which has different operation modes or forms governed by a Markov process.This class of systems can be used to represent complex real systems, which may experience abrupt variation in their structures and parameters.It is well known that random form switching can give rise to unexpected phenomena of the whole MJSs, even if the dynamics in each form is simple and well understood.Development of methodology for analysis of such systems is thus much deserving.
MJSs have been studied with growing interest and activity in recent years since they are first introduced by Kats and Krasovskii [1].For dynamics analysis, several important criteria for stochastic stability of MJSs have been established [2,3].The stationary response of the stochastically excited nonlinear MJSs is studied by Huan et al. [4].For the problem of optimal control, Krasosvkii and Lidskii studied the LQR control of Markovian jump linear systems firstly [5].As a consequence, considerable attention has been paid to the optimal control problem of MJSs [6,7].Sworder solved the jump linear Gaussian problem for finite time horizon [8].The infinite time horizon problem of linear MJSs was studied by Wonham [9] and Fragoso and Hemerly [10].The optimal control solution for linear MJSs subject to Gaussian input and measurement noise is given by Ji and Chizeck [11].Ghosh et al. investigated the ergodic control problem of switching diffusion [12].However, most of the published results are applicable to linear MJSs.Far less is known about the optimal control of nonlinear MJSs, particularly for stochastically excited nonlinear MJSs.
Recently, a nonlinear optimal control strategy of stochastic excited nonlinear systems has been proposed by Zhu based on the stochastic averaging method [13] and stochastic dynamical programming principle.It is proved that this strategy is quite effective for response alleviation of nonlinear stochastic systems.The goal of this paper is to extend the strategy for nonlinear stochastic system to the one for nonlinear stochastic MJSs.
An infinite time optimal control problem for single degree-of-freedom (SDOF), stochastically excited, nonlinear, Markovian jump system is investigated in this paper.The organization of this paper is as follows.In Section 2, the equation of SDOF nonlinear Markovian jump system is examined.Stochastic averaging method is applied to this system in Section 3. The partially averaged equation of total energy for each form of the original system is obtained.A 2 Shock and Vibration finite set of coupled dynamical programming equations is then set up in Section 4, from which the optimal control force is determined.The Fokker-Planck-Kolmogorov (FPK) equation associated with the fully averaged Itô equation is also derived and solved.In Section 5, two examples are worked out in detail to illustrate the application and effectiveness of the proposed control strategy.
If random excitations   () and Markovian jump process () are ergodic, then the response of controlled system approaches to stationary and ergodic as   → ∞.In this case, the performance index (3) is rewritten as Equations ( 1) and ( 4) constitute the mathematical formulation of the infinite-horizon optimal control problem for a SDOF nonlinear system undergoing Markovian jump.In this paper, we consider the perfect observation case, which means that the system state ((), ẋ ()) and values of () are all available at each time .

Partially Averaged Equation
Let  =  and  = ẋ ; (1) can be converted to the following Itô stochastic differential equations [11,16]: where   () are independent standard Wiener processes;  and  are the generalized displacement and generalized momentum, respectively;   (, ()) = √2  ℎ  (, ()).The Hamiltonian function  (total energy) of the Hamiltonian system associated with system (5) is Suppose that () is a slow jump process and independent of system's state.That means that, for original system (1) or (5), the system jumps among the  forms or modes slowly and independently.We consider one of the forms by letting the Markovian jump process be arbitrarily fixed at () =  ( ∈ ).
When () = , denote  () (, ) = (, , ()),  () (, ) = (, , ()), and  ()  () =   (, ()).Since  is a function of  and , the Itô equation for  can be derived from (5) by using Itô differential rule as follows [17]: In the case of light damping and weak excitations, the Hamiltonian  in system ( 7) is a slowly varying process while the generalized displacement  in ( 5) is a rapidly varying process.Since the slowly varying process is essential for describing the long-term behavior of the system, the stochastic averaging method is used to average out the rapidly varying process and to yield the following partially averaged Itô equation for  [13,18,19]: where The region of integration is Ω = { | (, 0) ≤ }, and the parameter Note that the second term  ()  () on the right-hand of ( 8) is unknown since  () are unknown function of , , and  at this stage.
To be consistent with the partially averaged equation ( 11), performance index (4) is also partially averaged: that is, The original optimal control problem (1) and ( 4) has been converted to the partially averaged optimal control problem (11) and (12).The partially averaged optimal control problem is much simpler than the original one since ( 11) is only onedimensional.It has been proved [20] that the optimal control for the partially averaged control problem is quasi-optimal control for the original one.

Optimal Control Law
For the proposed control problem, we introduce the value function: According to the stochastic dynamical programming principle and Markovian jump rules, the following dynamical programming equation can be established [16]: where   are the transition rates described in (2).Equation ( 14) is a finite set of second-order partial differential equations indexed by .These equations are coupled through the zeroorder terms ∑  =1   (, ).The necessary condition for minimizing the left-hand side of ( 14) is Let the cost function  be of the form where (, ) ≥ 0, and  is a positive constant.By substituting ( 16) into (15) and exchanging the order of the derivative with respect to  and averaging in (15), the expression of the optimal control force is determined: Note that (, ) is generally a nonlinear functional of .
Hence,  * (, , ) is a nonlinear function of  and ; that is, the optimal feedback control is nonlinear.To ensure that the control force  * (, , ) is indeed optimal, the following sufficient condition should be satisfied: Substituting the expression for  * (, , ) in ( 17) into (14) yields the final dynamical programming equation: where The value function (, ) can be obtained by solving (19).The optimal control force  * (, , ) is then determined by substituting (, ) into expression (17).By inserting the optimal control force  * (, , ) into (11) and averaging the term ⟨ * (, , )(/)⟩, the following completely averaged Itô equation is obtained: The FPK equation associated with the fully averaged Itô equation ( 21) is [4,21,22] where (, , ) is the probability density function of total energy  for optimally controlled system (1) while the system operating in th form ((, , ) for uncontrolled system can be obtained by letting  * (, , ) = 0).The initial condition is and boundary conditions are The FPK equation ( 22) is a finite set of second-order partially differential equations, which are coupled through the zero-order terms.Equation (22) does not admit easy solution, analytically or numerically.However, in practical application we are more interested in the stationary solution of FPK equation (22).In this case, FPK equation ( 22) is reduced by letting / = 0.Then, the stationary probability density (, ) is obtained readily from solving (22) numerically.The stationary probability density () can be obtained from (, ) as follows: where  is a normalization constant.The stationary probability density () of the generalized displacement and the mean value [] of the total energy are then obtained as where is the stationary joint probability density of the generalized displacement and generalized momentum.

Numerical Example
To demonstrate the application and to assess the effectiveness of the proposed control strategy, numerical results are obtained for two examples.
Let  =  and  = ẋ .The Hamiltonian associated with system (28) is As a Hamiltonian system, (28) can be expressed as Upon stochastic averaging, (11) for system (28) is obtained, for which Ω = { | (, 0) ≤ }, and For the proposed control strategy, the partially averaged cost function  is of the form of ( 16) with Following ( 17), the expression of the optimal control force is Substituting  * (, , ) in (33) into the dynamical programming equation ( 14), completing the averaging yields the following final coupled dynamical programming equations: (34) To solve (34), the value function (, ) is supposed to be of the following form: Since the optimal control force  * is function of /, only the coefficients  1 () and  2 () are determined by inserting (32) and ( 35) into (34): Figure 1: Stationary probability density () of total energy of uncontrolled and optimally controlled system (28).The lines are obtained from numerical solution of (37) while the dots are obtained from direct simulation of original system (28).
By substituting (35) into (17) with the coefficients determined by (36), the optimal control force is determined.Then, the following completely averaged FPK equation is obtained: where Solving (37) numerically with / = 0, the stationary probability density (, ) of optimal controlled system can be obtained.The stationary probability densities () of the total energy, () of the generalized displacement, and the mean value [] of the total energy are then obtained using (25) and (26).
Some numerical results are obtained as shown in Figures 1-4 for system parameters  1 = 0.03;  2 = 0.03;  = 1.0; Figure 2: Stationary probability density () of displacement of uncontrolled and optimally controlled system (28).The lines are obtained from numerical solution of (37) while the dots are obtained from direct simulation of original system (28).
The stationary probability density () of the total energy for uncontrolled and optimally controlled system (28) is shown in Figure 1.Obviously, as  increases, () for optimally controlled system (28) decreases much faster than that for uncontrolled system.The stationary probability density () of the displacement is plotted in Figure 2 with the same parameters as in Figure 1.For optimally controlled system, there is a much higher probability that  will be located near their equilibrium.Hence, () has much larger mode and smaller dispersion around  = 0 for optimally controlled system than those for uncontrolled system.This implies that the proposed control strategy has high effectiveness for attenuating the response of system (28).
Figure 3 shows the mean value [] of the total energy of uncontrolled and optimally controlled system as functions of ratio  11 / 22 ( 22 =  33 ).Obviously, higher value of ratio  11 / 22 implies lower probability for the system operating in form () = 1 and higher probability for the system operating in forms () = 2 and () = 3.When the system operates in form () = 1, the system has the smallest damping coefficients and the largest amplitudes of stochastic excitations than those when it operates in forms () = 2 and () = 3.As a consequence, [] for uncontrolled system decreases monotonously while the ratio  11 / 22 increases, as shown in Figure 3.However, [] for optimally controlled system always keeps a very low value despite of the ratio  11 / 22 , which indicates that the proposed control strategy is very effective and quite robust with respect to the ratio  11 / 22 .In Figures 1-3, lines are obtained by numerical solution of (37), while dots are obtained by direct simulation of system (28).Observe that the dots match closely with the corresponding lines, demonstrating the validity and accuracy of the proposed method.The displacements of the uncontrolled and optimally controlled system (28) are shown in Figure 4, from which the effect of the optimal control on the displacement can be visualized intuitively.Upon the stochastic averaging method, the original jump system (40) can be approximately substituted by the partially averaged system (11) with

Example
2 ) Following (17), the expression of the optimal control force is Then, the following final dynamical programming equation is obtained: / can be obtained from solving (43) numerically.Then the optimal control force is determined by substituting / into (17).Substituting the optimal control force (17) into the partially averaged equation (11)    The stationary probability densities () of total energy and () of the displacement of uncontrolled and optimally controlled system (40) are evaluated and plotted in Figures 5 and 6, respectively.The mean value [] of total energy of uncontrolled and optimally controlled system (40) as functions of ratio  11 / 22 is shown in Figure 7.It is seen that the optimal control force alleviates the response of system (40) significantly.This effect can be seen more apparently from Figure 8. Again, the analytical results obtained from solving (44) match closely with those from digital simulation of original system (40).

Conclusions
In this paper, we proposed a nonlinear stochastic optimal control strategy for SDOF stochastically excited Markovian jump systems where the jump parameter takes on values in a finite discrete set.In the slow jump case, the original system was first reduced to one governed by a finite set of onedimensional partially averaged Itô equation for total energy upon stochastic averaging.Using the stochastic dynamical programming principle and Markovian jump rules, a finite set of coupled dynamical programming equations was derived, from which the optimal control force was obtained.The obtained optimal control force has been proved to be the

Figure 3 : 2 Figure 4 :
Figure3: Mean value [] of total energy of uncontrolled and optimally controlled system (28) as functions of the ratio  11 / 22 ( 22 =  33 ).The lines are obtained from numerical solution of (37) while the dots are obtained from direct simulation of original system (28).