Stabilizing Solution for a Discrete-Time Modified Algebraic Riccati Equation in Infinite Dimensions

We provide necessary and sufficient conditions for the existence of stabilizing solutions for a class of modified algebraic discretetime Riccati equations (MAREs) defined on ordered Banach spaces of sequences of linear and bounded operators. These MAREs arise in the study of linear quadratic (LQ) optimal control problems for infinite-dimensional discrete-time linear systems (DTLSs) affected simultaneously by multiplicative white noise (MN) andMarkovian jumps (MJs). Unlike most of the previous works, where the detectability and observability notions are key tools for studying the global solvability ofMAREs, in this paper the conditions of existence ofmean-square stabilizing solutions are given directly in terms of system parameters.Themethods we have used are based on the spectral theory of positive operators and the properties of trace class and compact operators. Our results generalise similar ones obtained for finite-dimensional MAREs associated with stochastic DTLSs without MJs. Also they complete and extend (in the autonomous case) former investigations concerning the existence of certain global solutions (as minimal, maximal, and stabilizing solutions) for generalized discrete-time Riccati type equations defined on infinite-dimensional ordered Banach spaces.


Introduction
In recent years, the study of optimal control problems associated with stochastic systems with Markovian regime switching is of particular interest to researchers due to their various applications in finance, biology, engineering, and so forth.Even in the case of linear systems, the optimization problems become considerably hard, when we are in infinite dimensions and/or the Markovian process has infinite state space (see, e.g., [1][2][3][4][5] and the references therein).As we know from the LQ optimal control theory of DTLSs with MN and MJs (see, e.g., [5][6][7]), the design of the optimal control is closely related to the existence of a stabilizing solution (SS) for an associated generalized discrete-time Riccati equation (GDTRE).In this paper, we investigate the solution properties for a class of MAREs of control associated with autonomous infinite-dimensional DTLSs with MN and infinite MJs.These MAREs are time-invariant versions of the infinitedimensional GDTREs studied in [8].For a detailed treatment of finite-dimensional GDTREs, the reader can consult [6,7] and the references therein.The problem of existence of SSs for infinite-dimensional GDTREs associated with stabilizable DTLSs with infinite MJs was investigated in [3,5,9,10] under either stochastic detectability or observability hypotheses.In [8], a set of necessary and sufficient conditions for the existence of SSs is expressed in terms of feasibility of some linear matrix inequalities (LMIs) system.
The general theory of GDTREs applies to our MAREs but new special results are strongly expected in the autonomous case.For example, in finite dimensions it is proved (see [11]) that necessary and sufficient conditions for the existence of stabilizing solutions for MAREs can be given directly in terms of system parameters.They consist in verifying whether the stochastic system is stabilizable and  = 1 is not an unobservable eigenvalue for a pair of associated operators.Although it is not always possible to give an infinite-dimensional version of a finite-dimensional result (as an example, we recall the researches proving that Hautus Lemma does not work in infinite dimensions [12,13]), in this paper we have tried to do this for Theorem 12 from [11].
The approach we propose is based on operator theory tools and the properties of positive operators and their adjoints.Using the results from [8], we prove that conditions similar to those in [11] are necessary for the existence of SSs for infinite-dimensional MAREs (see Theorem 10).To obtain sufficient conditions, we extend the class of eigenvalues  that must not be unobservable to the set [1, ∞) and we impose additional constraints for the coefficients of the DTLSs with MN and MJs (see Theorems 11 and 13).When being applied to finite-dimensional DTLSs with MN, our results recover the ones in [11].However, Corollary 14 seems to be new for finitedimensional DTLSs with finite Markovian jumps.As we have shown in Example 1, it provides necessary and sufficient conditions for the existence of SSs for MAREs associated with DTLSs with MJs which are neither stochastically observable nor stochastically detectable.In this case, the results like those from [2,5,10] do not work, while a LMIs approach would lead to a large number of operations for systems of large dimensions.
The paper is organized as follows.In Section 2, we introduce some basic notation and terminology and we formulate the problem.Section 3 briefly reminds us about certain properties of nuclear (trace class) and compact operators and extends to a more general framework, the well-known result: "the dual space of trace-class operators space is isomorphic to the set of bounded operators." Also, we define the notions of maximal and stabilizing solution for a MARE and we show how to apply the results from [8] to the MAREs discussed in this paper.In Section 4, we obtain the main results.We first introduce the notion of unobservable eigenvalue for a pair of operators, by using spectral properties of positive operators.Then, we provide necessary and sufficient conditions for the existence of stabilizing solutions to MAREs (see Theorems 10,11,and 13).To obtain the sufficient conditions from Theorems 11 and 13, we assume additional hypotheses like compactness of the coefficients or trace-class membership.Corollary 14 shows that the finite-dimensional results from [11] are direct consequences of Theorems 10,11,and 13.Section 5 is devoted to numerical examples.We study three examples, which cover the cases of MAREs associated with finite-dimensional DTLSs with finite MJs and infinitedimensional DTLSs with infinite MJs.As mentioned above, Example 1 shows the advantages of Corollary 14 over previously published results [2,5,10].
Finally, the last section provides conclusions and further research lines.

Notations and Statement of the Problem
Let , , and  be real separable Hilbert spaces.We denote by (, ) the real Banach space of linear and bounded operators from  into  and by () the Banach subspace of () := (, ), formed by all self-adjoint operators.As usual * denotes either the adjoint of a linear and bounded operator or the dual of a Banach space.We will write ⟨⋅, ⋅⟩ for the inner product and ‖ ⋅ ‖ for norms of elements and operators, unless otherwise is indicated.An operator  ∈ () is called nonnegative and we write  ≥ 0, if  is selfadjoint and ⟨, ⟩ ≥ 0 for all  ∈ .
Let Z be an interval of integers, which may be finite or infinite.If (, ‖ ⋅ ‖) is a real Banach space, then  Z  = { = {() ∈ } ∈Z , ‖‖ Z = sup ∈Z ‖()‖ < ∞} is a real linear space with the usual termwise addition and (real) scalar multiplication.Moreover,  Z  is a Banach space when endowed with the norm ‖ ⋅ ‖ Z .If  is (, ) or (), then  Z  will be denoted by  Z (,) or  Z () .Let N be the set of natural numbers and N * = N − {0}.We denote by (N,  Z () ) the Banach space  I  Z

𝑆(𝐻)
obtained from  Z  by replacing Z and  by N and  Z () , respectively.An element  ∈  Z () is said to be nonnegative (we write  ⪰ 0) iff () ≥ 0 for all  ∈ Z.The cone K Z  of all nonnegative elements of  Z () introduces the following order on  Z () : Let   be the identity operator on  and let Φ  be the element {Φ  () =   } ∈Z of  Z () .We say that  ∈  Z () is positive and we write  ≻≻ 0, if there is for all  ∈ N and uniformly positive (we write   ≻≻ 0) if there is  > 0 such that   ⪰ Φ  for all  ∈ N.
Let E :  Z () →  Z () , and As we know from [5,7], (O) has a solution  opt if the following modified algebraic Riccati equation (MARE): has a mean-square stabilizing solution (see Definition 8).
Our problem is to provide a set of necessary and sufficient conditions for the existence of a stabilizing solution for MARE (10).

Sequences of Nuclear and Compact Operators.
As in [9], we denote by  1 () the Banach space of all nuclear operators from ().It is known that  1 () is a Banach space when endowed with the nuclear norm ‖‖ 1 = Tr[ √  * ].Here, Tr[⋅] is the trace operator.It is well know (see, e.g., [14,15]) that For further properties of nuclear operators, the reader is referred to [9,[14][15][16].It is known that the linear subspace is a Banach space (see [9]) when endowed with the norm ‖| ⋅ |‖ 1 .An easy computation (see [9]) shows that In the sequel we sometimes use a special element of N  , defined for any  ∈ Z and  ∈  by Here,  ⊗  ∈  1 () is defined by  ⊗ () = ⟨, ⟩.
Let C  denote the set all compact operators from ().Then, C Z  will be the subset of  Z () formed by sequences of compact operators; that is, It is well known that  1 () is dense in C  in the uniform operator topology (see, e.g., Problem 5.69 from [15]).The following lemma, whose proof is a simple exercise for the reader, states that a similar result remains true for finite sequences of nuclear and compact operators.
Remark 2. In the case when Z is infinite, the above result is not true as it is proved by the following counter example.Let  = R and let Z = N.Then, Proposition 3. The dual space (N  ) * of the Banach space N  is isometrically isomorphic with  Z () .The isomorphism  :  Z () → (N  ) * is defined by () =   , where   is the linear functional: Proof.Let  ∈  Z () and  ∈ N  be arbitrarily chosen.The following inequalities: show that   ∈ (N  ) * and           ≤ ‖‖ Z .

Maximal and Stabilizing Solutions of MARE.
In this section, we recall some definitions and results which we frequently use in the rest of the paper.To be consistent with previous publications (see, e.g., [7,8] and the references therein) concerning discrete-time Riccati equations associated with DTLSs with MN and MJs, we introduce the following notation.For all  ∈  Z () , let We see that ) is a positive operator.Then, MARE (10) can be equivalently rewritten as where, for all Following [8], we define the dissipation operator and the subsets Γ Σ and ΓΣ of (N,  Z () ) Lemma 28 from [8] (which is an operatorial version of the Schur complement Lemma) ensures that  ∈ ΓΣ iff (if and only if) R() −  ≻≻ 0. Definition 5. (a) We say that  ∈  Z () is a solution of MARE (10) if ( + Π 3 )() is invertible for all  ∈ Z and  satisfies (10).

Proposition 7. Assume (P1). If there is 𝑊 ∈ 𝑙 Z
(,) such that   (,  0 ) is exponentially stable, then MARE (10) has a maximal solution  max which has the property that Theorems 9 and 13 from [8] ensure that, in the time invariant case, the notion of stabilizing solution of MARE (10) from [8] is equivalent with the one introduced by the following.
Definition 8.By a stabilizing solution of MARE (10), we mean a solution X ∈  Z () with the property that the evolution operator   X (,  0 ),  ≥  0 , ,  0 ∈ N, where is exponentially stable.
Finally, we recall (see (4.8) in [8]) that mapping R can be equivalently rewritten as for all  ∈  Z (,) .We will use later this formula for different values of .

Main Results
In this section we establish necessary (see Theorem 10) and sufficient conditions (see Theorems 11 and 13) for the existence of a stabilizing solution to MARE (10).They are related to the nonexistence of certain unobservable eigenvalues for a pair of operators which are defined with the coefficients of (10).For finite-dimensional MAREs, we recover the results from [17] (see also [11] for the continuous time case).Unlike [17], our results apply to stochastic systems with Markovian jumps.
In the case when Z is finite, we have the following converse of Theorem 10.

Theorem 11. Assume that Z is finite and (P1) and (P2) hold. If
(a) Π is stabilizable and ) has not an unobservable eigenvalue  ≥ 1, then the algebraic Riccati equation (10) has a stabilizing solution.
Proof.From Proposition 7, we know that the algebraic Riccati equation ( 10) has a maximal solution  max .Assume by contradiction that  max is not stabilizing; that is,    max (,  0 ) is not exponentially stable.Then,  Π   max ≥ 1 and there is . For the sake of simplicity, let us denote  Π   max by .

Let us prove that 𝑦
for all  ∈ N  .Rewriting (33) for  replaced by   max , we see that the maximal solution  max of the algebraic Riccati equation (10) satisfies the following equation: Hence,    max = −(Π   max ( max ) −  max ).From (P1)(i) and (29),    max ⪰ 0. Using the properties of  * , we obtain It follows that  * (   max ) = 0.An easy computation shows that Lemma 28 from [8] (a Schur complement result for sequences of operators) and (P1)(i) imply that  − and , it follows easily that We know that, for any  ∈ () and {  } ∈N an orthonormal basis of , the increasing sequence   = ∑ ∞ =1 (  )⊗ (  ) ∈  1 () is weakly convergent to .It is not difficult to see that this property remains valid for each component () of any  ∈  Z () .So there is a nonnegative, increasing sequence   ∈ N  ,  ∈ N which is componentwise weakly convergent to .The positiveness of  * and the inequality we can apply (40) and we get for all  ∈ Z. Now let us recall the following property of nonnegative operators: Tr[ * ] = 0 ⇒  = 0. From (47), we obtain (48) Passing to the limit as  → ∞ in the above equality, we get Taking into account that () is invertible for all  ∈ Z, we deduce that √()( for all X ∈ N  .In view of (40), we get for all X ∈ N  .Let us prove that the above equality remains true if we replace X with an arbitrary  ∈  Z () .As mentioned above, there is a sequence X ∈ N  that converges componentwise and weakly to .That is, X () →  → ∞ (), weakly, for all  ∈ Z. From hypothesis (P2), it follows that for all  ∈ Z and  = 0, . . ., .This is because the multiplication to the right and to the left with compact operators promotes the weak convergence of operators to the uniform convergence. Therefore, and passing to the limit for  → ∞ we obtain In view of (45), we just have obtained a contradiction of (b).Therefore,  max is a stabilizing solution and the conclusion follows.
Remark 12.If the maximal solution would have the property that  Π   max ≤ 1, the proof of the above theorem could be modified such that condition  ≥ 1 is to be replaced by  = 1.
Before stating the next result, we note that ,  ∈ N  for all  ∈ N  and  ∈  Z () ; that is, N  is a two-sided ideal of  Z () .
In the case when Z is infinite, we have the following version of Theorem 11.

Theorem 13. Assume that Z is infinite and 𝐴
) has not an unobservable eigenvalue  ≥ 1, then the algebraic Riccati equation (10) has a stabilizing solution.
Proof.Arguing as in the proof of the above theorem, we see that (10) has a maximal solution  max .Since for all  ∈  Z () and N  is a two-sided ideal of  Z () , we deduce easily that for all  ∈  Z () .Assuming by contradiction that  max is not a stabilizing solution and reasoning as in the proof of the above theorem, we deduce that  Π   max ≥ 1 and there is , which contradicts the assumption  * ̸ = 0.As in the proof of Theorem 11, we obtain  * ( − −1  [ * ] ) = 0 and Letting  =  −1 ( * | N  ) and repeating step by step the arguments in the proof of Theorem 11, we obtain successively √()( ] for all  ∈  Z () and ( 54) holds.Therefore, we have obtained a contradiction of statement (b).Consequently,  max is a stabilizing solution and the proof is complete.Now, let us apply our results to finite-dimensional MAREs associated with stochastic systems of the forms ( 5) and ( 6).We will prove that conditions (a) and (b) from Theorem 13 are necessary and sufficient for the existence of a stabilizing solution to MARE (10).Corollary 14. Assume that Z is finite and the Hilbert spaces  and  are finite-dimensional.The algebraic Riccati equation (10) has a stabilizing solution if and only if Proof.We first recall that, in finite dimensions, all linear and bounded operators are compact and nuclear.Also, if Π is stabilizable, the maximal solution of MARE (10) has the property  Π   max ≤ 1.Indeed, from the proof of Theorem 9 from [8], we know that  max () is the strong limit of an increasing sequence   (),  ∈ Z, of solutions of certain associated Lyapunov equations.Denoting by Π  1 the Lyapunov operators associated with these equations, we also know that Π  1 converges to Π   max and  Π  1 < 1 (see [7,8], e.g.,).Since the eigenvalues of a matrix and consequently its spectral radius depend continuously on the matrix coefficients, we pass to the limit as  → ∞ in the last inequality and we get  Π   max ≤ 1.The conclusion follows.
Further, we observe that the hypotheses of Theorems 10 and 11 are fulfilled.The necessity part of the corollary follows from Theorem 10 while the sufficiency part is a direct consequence of Theorem 11 and Remark 12.

Numerical Examples
In this section, we provide some numerical examples which show the efficiency of our theory.Even in the finitedimensional case our results (see Corollary 14) seem to be new when applied to DTLSs with Markovian jumps.So, let us begin with a finite-dimensional example which proves that Corollary 14 is a viable alternative when stochastic detectability and stochastic observability conditions fail to hold.
Let us solve the optimization problem (O) defined in Section 1.
In this special case, problem (O) is exactly a linear quadratic control problem associated with (5), (58).The associated MARE (10)  (61) We know [7] that if MAREs (60) and (61) have a stabilizing solution X, then the optimal control problem (O) has a solution () =  X (())(), where  X is defined by (32).So we only have to find a stabilizing solution of (60) and (61).
Let () be the stochastic system (5) without control (i.e.,   = 0,  = 0, . . ., ) and let (, ) be the system defined by () and the output (58).The existing literature results (see [7] and the references therein) show that (10) has a stabilizing solution if the stochastic system with control is stabilizable and if (, ) is either stochastically observable or stochastically detectable.We will establish that (, ) is neither stochastically uniformly observable nor stochastically detectable.(Note that stochastic observability does not imply stochastic detectability [7].)In this case, we will see that Corollary 14 ensures the existence of the stabilizing solution.
Proposition 14 from [10] ensures that this stabilizing solution is nonnegative and maximal among all nonnegative solutions of (60).Then, solving (60) and choosing the maximal solution, we get the stabilizing solution X.
The following example is an application of Theorem 10.

Conclusions and Further Research
In this paper we have obtained necessary and sufficient conditions for the existence of stabilizing solutions for MARE (10) in infinite dimensions.These conditions are similar to those given in [11,17] for finite-dimensional MAREs and do not involve other detectability and observability conditions.They are viable alternatives to existing results (see, e.g., [3,5,7] and the references therein) obtained under detectability or observability hypotheses.
The main difficulties in obtaining a "perfect" analog of the results from [17] (or [11]) are related to the fact that Lemma 1 is not true for infinite Z and we cannot ensure the existence of a nonnegative linear functional  * satisfying (34) and condition  * (N  ) ̸ = {0}.To compensate these gaps, we have assumed a compactness hypothesis in Theorem 11 and the condition   ,   ∈ N  ,  = 0, . . ., , in Theorem 13.A natural question is how to relax these conditions.
Another open problem is whether the assertion  Π   max ≤ 1, where  max is the maximal solution of MARE (10), remains true in infinite dimensions or for infinite Z.The validity of this assertion will improve Theorems 11 and 13, as indicated in Remark 12.
Also, further research is required to study the existence of stabilizing solutions for MARE (10) in the case when the state space of the Markov chain is a general Borel Space as in [1].