Stability of a Class of Stochastic Nonlinear Systems with Markovian Switching

This paper investigates the stability of a class of stochastic nonlinear systems with Markovian switching via output-feedback. Based on the backstepping design method and homogeneous domination technique, an output-feedback controller is constructed to guarantee that the closed-loop system has a unique solution and is almost surely asymptotically stable.The efficiency of the outputfeedback controller is demonstrated by a simulation example.


Introduction
There are lots of real systems, such as hierarchical control of manufacturing systems, financial engineering, and wireless communications systems, whose structure and parameters may change abruptly.Further, if the occurrence of these events is governed by a Markov chain, these systems are called Markovian jump systems.As one branch of modern control theory, the study of Markovian jump systems has aroused lots of attention with fruitful results achieved for linear case, such as the controllability and observability [1], the stability and stabilization [2][3][4],  2 control [5],  ∞ control [6,7], filtering [8], and model reduction [9].For semilinear stochastic differential equations with Markovian switching, [10] discusses the stabilization problem; [11] discusses the exponential stability problem for general nonlinear differential equations with Markovian switching.References [12,13] focus on the controller design for hybrid systems with the global Lipschitz condition or linear growth condition.Based on the backstepping design method developed by [14][15][16][17] investigates the control of stochastic systems with Markovian switching.
Considering that the system states are incompletely measurable, the problem of output-feedback control is more important and challenging than that of the state-feedback control in practical applications.Reference [18] addresses the problem of global output-feedback and link position tracking control of robot manipulators despite the fact that only link position measurements are available in the presence of incomplete model information.Reference [19] presents the output-feedback tracking controllers design for an underactuated ship and introduces global nonlinear coordinate changes to transform the ship dynamics to a system affine in the ship velocities to design observers to globally exponentially estimate unmeasured velocities.Reference [20] focuses on the problem of output-feedback tracking control for stochastic Lagrangian systems with the unmeasurable velocity.By using the structural properties of Lagrangian systems, a reduced-order observer is skillfully constructed to estimate the velocity.Inspired by [17], this paper aims to solve the output-feedback stabilization problem for a class of stochastic nonlinear systems with Markovian switching.As demonstrated by [21], due to the fact that these classes of systems' Jacobian linearizations are neither controllable nor feedback linearizable, the existing design tools are hardly applicable.
Compared with the existing results, the contributions of this paper are as follows: (1) The results in [22] consider the stabilization problem of stochastic nonlinear systems without considering Markovian switching.However, considering that systems may often undergo abrupt disturbances in the practical environment which can be modelled (2) Since the drift terms and diffusions terms are all Markovian switching, how to design an effective observer to deal with the unmeasurable states and how to design a control to guarantee that the closedloop system has a unique solution and is almost surely asymptotically stable are nontrivial work.
The remaining part of this paper is organized as follows.Section 2 offers some preliminary results.The problem investigated is described in Section 3.After that, in Section 4, the output-feedback controller is designed followed by a simulation example to show the effectiveness of the designed controller in Section 5. Finally, the paper is concluded in Section 6.

Preliminary Results and Useful Lemmas
The following notations will be used throughout this paper.R + denotes the set of all nonnegative real numbers, and R  denotes the real -dimensional space: R ≥1 odd ≜ { ∈ R :  ≥ 1 and  is a ratio of odd integers}.One has   = ( 1 , . ..,   )  ,  1 =  1 , and   = .For a given vector or matrix ,   denotes its transpose, Tr{} denotes its trace when  is square, and || is the Euclidean norm of a vector .C  denotes the set of all functions with continuous th partial derivatives.
Consider the stochastic differential equations with Markovian switching:  () =  ( () , ,  ())  +  ( () , ,  ()) , (1) where () ∈ R  is the state of system;  is an -dimensional independent standard Wiener process defined on the complete probability space (Ω, F, F  , ) with a filtration F  satisfying the usual conditions (i.e., it is increasing and right continuous while F 0 contains all -null sets).Let () be a right-continuous homogeneous Markov process on the probability space taking values in a finite state space  = {1, 2, . . ., N} with generator Γ = ( pq ) N×N given by for any ,  ≥ 0.Here  pq > 0 is the transition rate from p to q if p ̸ = q while We assume that the Markov process () is independent of the Wiener process ().The Borel measurable functions  : R  × R + ×  → R  and  : R  × R + ×  → R × are locally Lipschitz in  ∈ R  for all  ≥ 0.

Output-Feedback Stabilization of System (9)
By introducing the coordinates where  1 = 0,   = ( −1 + 1)/ −1 , and  > 1 is a constant to be designed, with (11), system (9) can be written as whose nominal nonlinear system is The design of output-feedback controller for system (9) is divided into three steps.In Step 1, one supposes that the states are available for measurement, and a state-feedback controller is designed for nominal nonlinear system (13).Then in Step 2, by constructing a reduced-order observer, an output-feedback controller is designed for (13).Finally, by using the homogeneous domination technique, the outputfeedback stabilization problem is solved for system (9).
Hence, (25) implies that the closed-loop system described by the compact form is globally asymptotically stable, where By introducing the dilation weight we know that (26) is homogeneous of degree .
(2) For any  0 ∈ R  and  0 ∈ , the solution of the closedloop systems is almost surely asymptotically stable.
For any  > 0, define the first exit time: (1) This paper is the first result about the output-feedback control of stochastic nonlinear systems with Markovian switching and uncontrollable linearizations.
(2) Since the drift terms and diffusions terms are all Markovian switching, a homogeneous domination approach is developed in this paper, which can effectively deal with the Markovian switching and uncontrollable linearizations simultaneously.

Conclusions
This paper investigates the output-feedback stabilization of stochastic nonlinear systems with Markovian switching for the first time.By using the backstepping design method and homogeneous domination technique, an output-feedback controller is constructed to guarantee that the closed-loop system is almost surely asymptotically stable.
There are some related problems for further consideration, for example, how to generalize the results in this paper to more general stochastic nonlinear systems with plant and parameter uncertainties.