Algorithms to Solve Stochastic 𝐻 2 /𝐻 ∞ Control with State-Dependent Noise

This paper is concerned with the algorithms which solve 𝐻 2 /𝐻 ∞ control problems of stochastic systems with state-dependent noise. Firstly, the algorithms for the finite and infinite horizon 𝐻 2 /𝐻 ∞ control of discrete-time stochastic systems are reviewed and studied. Secondly, two algorithms are proposed for the finite and infinite horizon 𝐻 2 /𝐻 ∞ control of continuous-time stochastic systems, respectively. Finally, several numerical examples are presented to show the effectiveness of the algorithms.


Introduction
Mixed  2 / ∞ control is an important robust control method and has been extensively investigated by many researchers [1][2][3][4].Compared with the sole  ∞ control, the mixed  2 / ∞ control is more attractive in engineering practice [4], since the former is a worst-case design which tends to be conservative while the latter minimizes the average performance with a guaranteed worst-case performance.Recently, stochastic  2 / ∞ control for continuous-and discrete-time systems with multiplicative noise has become a popular topic and has attracted a lot of attention [5][6][7].In [5], the finite and infinite horizon  2 / ∞ control problems were discussed for continuous-time stochastic systems with state-dependent noise.The finite and infinite horizon  2 / ∞ control problems were solved for discrete-time stochastic systems with state and disturbance dependent noise by [6] and [7], respectively.Moreover, mixed  2 / ∞ control was widely studied for stochastic systems with Markov jumps and multiplicative noise [8][9][10][11] due to their powerful modeling ability in many fields [12,13].
Generally, the existence of a  2 / ∞ controller is equivalent to the solvability of several coupled matrix-valued equations.However, it is difficult to solve these coupled matrixvalued equations analytically.Several numerical algorithms have appeared in dealing with deterministic and stochastic  2 / ∞ control.In [1], the finite horizon  2 / ∞ controller for continuous-time deterministic systems was obtained by using the Runge-Kutta integration procedure.In [14], an exact solution to the suboptimal deterministic  2 / ∞ control problem was studied via convex optimization.Two iterative algorithms were proposed for finite and infinite horizon  2 / ∞ control of discrete-time stochastic systems in [6] and [7], respectively.In [15], an iterative algorithm was proposed to solve a kind of stochastic algebraic Riccati equation in LQ zero-sum game problems.
However, most of these algorithms were concerned with the  2 / ∞ control for discrete-time systems.Up to now, the algorithm for stochastic  2 / ∞ control of continuous-time systems has received little research attention.This is because the coupled matrix-valued equations for the continuoustime  2 / ∞ control cannot be solved by recursive algorithms as in the discrete-time case.In this paper, we will study the algorithms to solve  2 / ∞ control problems for stochastic systems with state-dependent noise.Firstly, the algorithms for finite and infinite horizon  2 / ∞ control of discrete-time stochastic systems are reviewed.An iterative algorithm is presented to solve the infinite horizon  2 / ∞ control of discrete-time time-varying stochastic systems.For continuous-time stochastic systems, two algorithms are proposed for the finite and infinite horizon  2 / ∞ control, respectively.Some numerical examples are presented to illustrate the developed algorithms.
For conveniences, we make use of the following notations throughout this paper: R  : -dimensional Euclidean space; S  : the set of all × symmetric matrices;  > 0 ( ≥ 0):  is a positive definite (positive semidefinite) symmetric matrix;   : the transpose of a matrix ; : the identity matrix; Tr[]: the trace of matrix ; (): the mathematical expectation of .

Preliminaries
In this section, we will present some preliminary results for stochastic  2 / ∞ control, including the finite horizon case for discrete-time time-varying systems, the infinite horizon case for discrete-time time-invariant systems, the finite horizon case for continuous-time time-varying systems, and the infinite horizon case for continuous-time time-invariant systems.
Consider the following discrete-time time-invariant stochastic systems with state-dependent noise: Briefly, system (6) can be denoted by (, , ;  | ), and similar notations will be used in the following section.

if and only if the following coupled algebraic matrix-valued equations
Consider the following continuous-time time-varying stochastic system with state-dependent noise: where () ∈ R  , () ∈ R   , V() ∈ R  V , and () ∈ R   are, respectively, the system state, control, disturbance signal, and output.() is a standard one-dimensional Wiener process defined on the filtered probability space (Ω, F, F  , P) with F  = {() : 0 ≤  ≤ }. 0 is assumed to be deterministic for simplicity purposes.(), (), (), (), and () are matrix-valued continuous functions of suitable dimensions.

Discrete-Time Case
In [6,7], Zhang et al. provided the recursive algorithms to solve the coupled matrix-valued equations in Lemmas 1 and 2, respectively.Based on those results, this paper will present an algorithm to solve the infinite horizon  2 / ∞ control of discrete-time time-varying stochastic systems.
The following algorithm can be used to solve the coupled difference matrix-valued equations ( 2)-( 5) in Lemma 1 [6].
Algorithm 5. Consider the following.
The following algorithm can be used to solve the coupled algebraic matrix-valued equations ( 7)- (10) in Lemma 2 [7].Algorithm 6.Consider the following.
In [10], a necessary and sufficient condition for the infinite horizon  2 / ∞ control problem of discrete-time timevarying stochastic systems with Markov jumps was derived in terms of four coupled discrete-time Riccati equations.However, the Riccati equations in [10] were solved by trial and error and cannot be extended to the complicated case.The condition for the infinite horizon  2 / ∞ control of timevarying stochastic system (  ,   ,   ;   |   ) (or system (1)) is as follows.
Proof.This is a direct corollary of Theorem 2 in [10] and the proof is omitted.
In this paper, the essential difference between Lemmas 1 and 7 is that  is finite in the former while it is infinite in the later.Based on Algorithm 6, the coupled matrix-valued equations ( 17)-(20) can be solved by the following recursive algorithm.(ii) Compute the solution of these time-invariant matrixvalued equations by using Algorithm 6.
It is difficult for Algorithm 8 to compute all the solutions as  → ∞ for general time-varying system.However, it is easy to verify that the solutions of ( 17)-(20) are also periodic for periodic systems.Hence, Algorithm 8 is suitable for the periodic case, which will be shown by Example 1.

Continuous-Time Case
In contrast to the discrete-time case, it is more difficult to deal with the continuous-time stochastic  2 / ∞ control in Lemmas 3 and 4. In this study, the Runge-Kutta integration procedure and the convex optimization approach are applied to solve the coupled matrix-valued equations in Lemmas 3 and 4, respectively.
In Lemma 3, the coupled differential matrix-valued equations ( 12) and ( 13) can be viewed as a set of backward differential equations with known terminal conditions, which can be solved by the Runge-Kutta integration procedure [1].The following algorithm can be used to solve (12) and ( 13) in Lemma 3. Algorithm 9. Consider the following.
(ii) Solve this set of equations by using the Runge-Kutta integration procedure.
(iii) If the solutions of the set of equations are convergent, then the finite horizon  2 / ∞ control problem is solvable.Otherwise, the problem is unsolvable.
Next, we will study the algorithm for the solution of coupled algebraic matrix-valued equations ( 15) and ( 16) in Lemma 4. In the scalar case, the curves represented by ( 15) and ( 16) can be plotted in a ( 1 ,  2 )-plane, and the intersections of these curves, if they exist, are the solutions of ( 15) and ( 16).Moreover, the intersection in the second quadrant is the solution that we need, which will be shown in Example 2.
In the high-dimensional case, a suboptimal  2 / ∞ controller design algorithm for Lemma 4 was obtained in [8] by solving a convex optimization problem.However, this algorithm was developed under the assumption  1 = − 2 which was very conservative.Rewrite ( 15) and ( 16) as From the above, it can be seen that one matrix  2 cannot satisfy two different equations simultaneously expect in some very special cases.
In this paper, we try to present another convex optimization algorithm to solve (15) and (16).By Theorem 10 of [16], ( 1 ,  2 ) ∈ S  × S  is the optimal solution to max Since we have Θ 1 ≥ 0 and Θ 2 ≥ 0 if respectively.According to Schur's complement lemma, Θ 1 ≥ 0 and Θ 2 ≥ 0 are, respectively, equivalent to with Since (26) are linear matrix inequalities (LMIs), a suboptimal solution to coupled matrix-valued equations (21) may be derived by solving the following convex optimization problem: Moreover, the infinite horizon  2 / ∞ control problem of system ( 14) has a pair of solutions: Summarizing the above, the following algorithm can be used to solve (15) and ( 16) in Lemma 4.
Algorithm 10.Consider the following.
Remark 11.Note that, in Algorithm 10, conditions (26) are given in terms of linear matrix inequalities; therefore, by using the Matlab LMI-Toolbox, it is straightforward to check the feasibility of the convex optimization problem (28) without tuning any parameters.In fact, Algorithm 10 is also a suboptimal algorithm, and the conservatism comes from the inequality transforms (24).
Remark 12.In this paper, we consider the  2 / ∞ control for stochastic systems with only state-dependent noise.As discussed in [17,18], for most natural phenomena described by Itô stochastic systems, not only state but also control input or external disturbance maybe corrupted by noise.Therefore, it is necessary to study stochastic systems with state, control, and disturbance-dependent noise which makes the conditions for  2 / ∞ control more complicated.Searching for the numerical solutions for these conditions deserves further study.

Numerical Examples
In this section, several numerical examples will be provided to illustrate the effectiveness of Algorithms 8-10.
Example 1.Consider the infinite horizon  2 / ∞ control for two-dimensional periodic stochastic systems (1) with the following parameters: Apparently, the period of this system is  = 2.
By setting  = 1.8 and applying Algorithm 8, the evolutions of  1 ,  2 ,  1 ,  2 ,  = 1, 2 are illustrated in Figures 1  and 2, respectively, which clearly show the convergence of the algorithm.According to Algorithm 9, coupled differential equations ( 12) and ( 13) can be viewed as the following set of equations with known terminal conditions: Setting  = 0.4 and using the Runge-Kutta integration procedure, the evolutions of  1 (),  2 (),  1 (),  2 () are given in Figure 3, which clearly show the convergence of the solutions of ( 12) and ( 13).
have been reviewed and studied, and algorithms in the continuous-time case have been developed.The validity of the obtained algorithms has been verified by numerical examples.This subject yields many interesting and challenging topics.For example, how can we design numerical algorithms to solve the  2 / ∞ control problems of stochastic systems with state, control, and disturbance-dependent noise?This issue deserves further research.

Figure 4 :Example 3 .
Figure 4: The intersection in the second quadrant of curves represented by (15) and (16) in Example 2.