Lyapunov Characterization for the Stability of Stochastic Control Systems

Lyapunov-like characterization for the problem of input-to-state stability in the probability of nonautonomous stochastic control systems is established. We extend the well-known Artstein-Sontag theorem to derive the necessary and sufficient conditions for the input-to-state stabilization of stochastic control systems. Illustrating example is provided.


Introduction
The stabilization of various types of nonlinear systems has been widely studied in the past years, see, for instance, [1][2][3][4][5].The necessary and sufficient conditions for input-to-state stability and robust stability at the equilibrium state of nonlinear system are provided by Sontag and Wang [1].Angeli et al. [2] showed that there exists a positive definite Lyapunov function whose derivative along the system is a negative definite that guarantees a time-varying system satisfying in the integral input-to-state stability property.Later, Grune [3] derived a suitable Lyapunov function and established an input-to-state dynamical stability property for a timevarying system.The necessary and sufficient conditions for input-to-state stability of nonlinear time-varying system have been provided by Karafyllis and Tsinias [4].Ning et al. [5] employed an indefinite Lyapunov function rather than a negative definite function and established input-to-state stability and integral input-to-state stability of time-varying system.
The main purpose of this paper is to establish a Lyapunov characterization for the problem of ISSP of nonautonomous stochastic control systems (NSCS).We extend the wellknown Artstein-Sontag theorem (see [14,15]) established in Karafyllis and Tsinias [4] to the concept of stochastic control Lyapunov function (CLF) in order to derive the necessary and sufficient conditions for the ISSP of NSCS.We also establish the existence of an explicit formula of a feedback law exhibiting ISSP property and give some applications to feedback stabilization.The analysis used in this paper is closely related to that of Karafyllis and Tsinias [4].
The paper is organized as follows.In Section 2, we introduce the class of stochastic systems and some basic definitions and results that we are dealing with in this paper.We also describe a wider class of NSCS and we focus on the properties of stochastic CLF which play an important role in ISSP property.Finally, in Section 3, we state and prove the main results of the paper on the ISSP property of NSCS.We also provide a numerical example to illustrate our results.

Fundamental Definitions and Results
The purpose of this section is to introduce the notion of robust stability in probability (RSP) and ISSP property for a class of stochastic systems.For a detailed presentation of stochastic stability theory, we refer the reader to the book of Khasminskii [16] and the paper of Abedi et al. [11].
Definition 2. The equilibrium   ≡ 0 of the system (1) is (i) globally stable in probability, if, for every  > 0 and input V(), there exists a class -function (⋅) such that (ii) globally asymptotically stable in probability, if it is globally stable in probability and In the following, we recall the stochastic version of La Salle's invariant theorem established by Kushner [17].

Theorem 3. Suppose that there exists a Lyapunov function Φ defined on 𝑅
where D is the infinitesimal generator of the stochastic process solution of stochastic system (1) as follows: Then, the equilibrium solution   ≡ 0 of stochastic system (1) tends to the largest invariant set whose support is contained in the locus DΦ(  ) = 0 for any  ≥ 0 with probability 1.
We will now turn the attention to a wider class of NSCS and focus on the properties of stochastic CLF which play an important role in ISSP property in Section 3.
We recall the definition of RSP introduced by Abedi and Leong [18] on the neighborhood  ⊂   of the origin as follows.
Definition 4. The NSCS ( 7) is said to be RSP if there exists a neighborhood  of the origin in   and a function  :  + ×  →   , with (, 0) = 0, such that (i) for every  remains in , for all  ≥ 0, the solution   of the resulting closed-loop system where (  ) ≥0 is a standard   -valued Wiener process defined on a complete probability space (Ω, , ), is uniquely defined with V as input and (ii) the equilibrium solution   ≡ 0 of the resulting closed-loop system ( 8) is GASP with V as input.
Consider the NSCS (7), under a slight change of hypothesis in the notion of ISS property introduced by Karafyllis and Tsinias [4], we obtain the notion of ISSP in terms of Lyapunov functions as follows.
Definition 5. Let (, ) :  + × + →  + be a positive definite function, which is  0 , locally Lipschitz in ; the NSCS (7) satisfies the weak ISSP (wISSP) from the input V with gain (⋅) if each solution of NSCS (7) exists for all  ≥  0 and satisfies conditions (3) and (4) of Definition 2 provided that Definition 6.We say that NSCS (7) satisfies the ISSP from the input V with gain (⋅) if the following conditions hold: (i) the NSCS (7) satisfies wISSP property and As in the deterministic case (see [4,19]), we can easily establish the following elementary result.Proof.The proof of this lemma is a direct consequence of Definitions 2, 4, and 5 and the fact that each solution () of the NSCS (10) that corresponds to some V(⋅) coincides with the solution of the NSCS (7) with the same initial  0 and  0 and corresponding to V = (, ||), namely, satisfying (9).Conversely, each solution () of the NSCS (7) under restriction ( 9) is a solution of the NSCS (10) with input  = V/(, ||) and the same initial value.
The main results of this paper (Theorems 11 and 12) constitute the extensions of the well-known Artstein-Sontag theorem established in Karafyllis and Tsinias [4] and guarantee the existence of a  ∞ mapping  = (, ) in such a way that the resulting closed-loop system (8) satisfies ISSP property with V as input.
We denote by D the infinitesimal generator of the stochastic process solution of NSCS (7); that is, D is the second-order differential operator defined for any function Φ in  1,2 ( + ×   , ) by where 1 ≤  and  ≤ .We also denote by D  , 1 ≤  ≤  the first-order differential operator defined for any function Φ in  1 ( + ×   , ) by The following definition is an extension of Definition 2.4 established in [11] and which described the stochastic CLF that was used for ISSP of NSCS (7) at the origin.Definition 8. Let (, ) :  + ×  + →  + be a positive definite function, which is  0 , locally Lipschitz in , the NSCS (7) admits a stochastic CLF, if there exists a  1,2 function Φ :  + ×   →  + , class  ∞ functions  1 ,  2 , and a positive definite function  :  + →  + such that, for all (, , ) ∈  + ×   × , the following conditions hold: Since the differential operator D appears in the condition ( 14) stated in Definition 8, the computations in the stochastic case are more tedious than those in the deterministic case introduced by Karafyllis and Tsinias [4].We extended the concept of CLF introduced by Karafyllis and Tsinias [4] and obtained the notion of stochastic CLF to Definition 8 that was used for ISSP of the NSCS (7) at the equilibrium state.Relationships between the concepts of ISSP given above and their characterization in terms of Lyapunov functions are developed in the next section.In order to establish our main results as in Theorem 11, we need the following technical theorem established in [18].The proof of this theorem was exposed in [18] and is therefore omitted.Theorem 9. Consider the NSCS (7).Then the following statements are equivalent: (i) there exists a  ∞ function  :  + ×   →   with (, 0) = 0 in such a way that the resulting closed-loop system (8) satisfies RSP property; (ii) there exists a  0 function  :  + ×   →   with (, 0) = 0 being locally Lipschitz in , in such a way that the resulting closed-loop system (8) satisfies RSP property; (iii) the NSCS (7) admits a stochastic CLF.

Main Results
We study the problem of finding continuous feedback law in order to derive necessary and sufficient conditions for ISSP of NSCS.For the NSCS (7), we extend the well-known Artstein-Sontag theorem established in [4] by introducing the concept of stochastic CLF (Theorem 11).This result is a useful tool for designing an explicit formula of a feedback law that exhibits ISSP of the resulting closed-loop system (8) at the equilibrium state (Theorem 12).
Proof.The proof of this proposition is a direct consequence of Theorem 9 and Lemma 7. Indeed, assume that the resulting closed-loop system (8) satisfies the wISSP property; then, according to Lemma 7, the resulting closed-loop system (8) satisfies the RSP property.Therefore, by Theorem 9, the NSCS (7) admits a stochastic CLF.
The input-to-state stability results proven in this paper use a technique which is a combination of Karafyllis and Tsinias [4] decomposition in deterministic case and Abedi et al. [11] decomposition in stochastic case.We use this decomposition and extend the existing input-to-state stability results.The following theorem is an extension of the wellknown Artstein-Sontag theorem established in [4] (Theorem 5.1 of [4]).The proof of this theorem is a stochastic analogue of the deterministic proof of Karafyllis and Tsinias [4].In the proof of this theorem, we use the stochastic versions of Artstein-Sontag theorem established in Abedi et al. [11] and La Salle's invariance theorem developed in [17].
Theorem 11.Consider the NSCS (7) and let (, ) :  + ×  + →  + be a positive definite function, which is  0 , locally Lipschitz in .Then the following statements are equivalent: (i) there exists a  ∞ function  :  + ×   →   with (, 0) = 0 such that the resulting closed-loop system (8) satisfies the wISSP property with gain (⋅) from the input V.It turns out that the resulting closed-loop system (8) satisfies the ISSP property; (ii) there exists a  0 function  :  + ×   →   with (, 0) = 0 for all  ≥ 0, being locally Lipschitz in , such that the resulting closed-loop system (8) satisfies the same property as that in statement (i); (iii) the NSCS (7) admits a stochastic CLF.
(ii → iii) Suppose that there exists a function (, ), such that the closed-loop system (8) satisfies the wISSP with gain (⋅) from the input V.According to Lemma 7, the resulting closed-loop system (8) satisfies the RSP property.Then, by Proposition 10 and the converse Lyapunov theorem (Theorem 2) established by Kushner [20], which provided the existence of a Lyapunov function in some neighborhoods of the origin, there exists a  1,2 function Φ :  + ×   →  + and a continuous and positive definite function (, ) such that (13) where D 0 is the infinitesimal generator of the resulting closed-loop system (8).The latter inequality implies (14) where (Φ(, )) = (, ) and the condition D  Φ(, ) = 0. Therefore, the requirement in Definition 8 holds and Φ(, ) is a stochastic CLF for the NSCS (7).
(iii → i) Assume that the NSCS (7) admits a stochastic CLF.Consider the functions  1 ,  2 , and Φ as defined in (13) and (14).From (13) From (17) and Φ(, ) ≥ 0, Φ  = Φ(, ) is a supermartingale.By a supermartingale inequality established by Rogers and Williams [21], for any class  ∞ function Γ(⋅), we have Let Φ 0 = Φ(0,  0 ).By using Ito formula for NSCS (7), we obtain By taking into account (17) and (19) we get From (20) we have Invoking ( 17) and ( 21) yields Hence, from ( 18) and ( 22), we have Thus, For a given  ∞ functions  1 ,  2 , and Γ, define and so, For a given  > 0, choose Γ(Φ  ) such that Thus, from (28) and Definition 4, we have that the equilibrium is globally stable in probability with respect to the resulting closed-loop system (8).On the other hand, from (17) and the stochastic version of La Salle's invariance Theorem 3, we obtain the stochastic process solution () of the resulting closed-loop system (8), which tends to 0 with probability 1; that is, Therefore, from (30) and the above global stability in probability and Definition 4, we get the equilibrium is RSP with respect to the resulting closed-loop system (8).The desired wISSP property for the resulting closed-loop system (8) is a consequence of Lemma 7.
The following theorem, which is an immediate consequence of Theorem 11, is a stochastic extension of Proposition 5.2 in [4].The proof of this theorem is a stochastic analogue of the deterministic proof of Karafyllis and Tsinias [4].In the proof of this theorem, we use an explicit formula of a feedback law derived by Florchinger and Verriest [7] in exhibiting wISSP property for the resulting closed-loop system (8).