Study on Stochastic Linear Quadratic Optimal Control with Quadratic and Mixed Terminal State Constraints

This paper studies the indefinite stochastic LQ control problem with quadratic and mixed terminal state equality constraints, which can be transformed into a mathematical programming problem. By means of the Lagrangian multiplier theorem and Riesz representation theorem, the main result given in this paper is the necessary condition for indefinite stochastic LQ control with quadratic and mixed terminal equality constraints. The result shows that the different terminal state constraints will cause the endpoint condition of the differential Riccati equation to be changed. It coincides with the indefinite stochastic LQ problem with linear terminal state constraint, so the result given in this paper can be viewed as the extension of the indefinite stochastic LQ problem with the linear terminal state equality constraint. In order to guarantee the existence and the uniqueness of the linear feedback control, a sufficient condition is also presented in the paper. A numerical example is presented at the end of the paper.


Introduction
Linear quadratic (LQ) control is an extremely important class of control problems in both theory and application.It is pioneered by Kalman [1] for deterministic systems and was extended to stochastic systems by Wonham [2].In recent years, extensive research has been carried out in the socalled indefinite stochastic LQ control, in which the cost weighting matrices are allowed to be indefinite; refer to [3][4][5][6] for detailed accounts.A basic assumption in the LQ theory, both for deterministic and stochastic cases, is that the variable is unconstrained except for the differential equations constraint.As far as we know, very few results for constrained deterministic LQ can be found compared with the unconstrained one, not to mention the stochastic LQ control [7].While in many real applications, constrained LQ control problem (such as nonnegativity and bound constraints for state and control variables) is a well-posed problem, constrained stochastic LQ control problem has a concrete application background, but the conventional LQ approach would collapse in the presence of any constraints.Study on the constrained stochastic LQ control will contribute to both theory and application a lot.
Huang and Zhang [8] studied the indefinite stochastic LQ control problem with linear terminal state equality constraints.Necessary and sufficient conditions for indefinite stochastic LQ control problems were investigated based on the Lagrangian multiplier theorem and Riesz representation theorem.The result showed that the linear feedback optimal control can be obtained by solving systems of algebraic and differential equations.The previous results on unconstrained indefinite stochastic LQ can be viewed as a specified case of the main theorem in that paper.
This paper studied the indefinite stochastic LQ control problem with quadratic terminal equality constraints and mixed constraints, which can be viewed as the extension of [8].When the terminal state constraint is quadratic, the feasible region defined by the terminal constraint is nonconvex and multiple local minima abound, which makes the problem more complex to locate the optimum consistently.Developing a deeper understanding of the problems, as well as efficient algorithms for solving them, will have a big impact in many applications.Another reason for the study of this problem is that the methods used for solving this type of problem can be used to solve more general constrained optimal control problems.By means of the Lagrangian multiplier theorem and Riesz representation theorem, the main result in this paper is the necessary condition for indefinite stochastic LQ control with quadratic terminal constraints and mixed terminal constraints.The result showed that the difference of the terminal state constraints will cause the endpoint condition to be changed in the differential equations we obtained for the linear constraint control problem, which coincides with the reality.In order to guarantee the existence and the uniqueness of the linear feedback control, a sufficient condition is also presented in the paper.Numerical example is presented at the end of this paper.

Problem Statement and Preliminaries
in which () ∈  × [0, ].The cost weight matrix in Problem 1 is not necessarily positive, which usually causes Problem 1 to be called indefinite stochastic LQ control problem with terminal constraint.First, we present some definitions and lemmas that will be used and then transform the optimal control Problem 1 into a deterministic control problem.
Definition 2 (Gateaux differential).Let  be a vector space, let  be a normed space, and let  be a (possibly nonlinear) transformation defined on a domain  ⊆  with a range  ⊆ .Given  ∈  ⊂ , and ℎ is an arbitrary vector in .If the limit exists, then it is called Gateaux differential at  with an increment ℎ.If the limit exists for arbitrary ℎ ∈ , the transformation  is called Gateaux differentiable at .
then  is said to be Frechet differentiable at , and (; ℎ) is said to be the Frechet differential of  at  with an increment ℎ.
And taking the place of () in ( 2), we have According to the Itô integrals formula for   , in which () is the solution of ( 8), we have Define (  ) = ; it is obvious that  is a symmetric matrix.Integrate both sides of ( 9) from 0 to  with variable , then compute the derivatives of both sides of (9) after taking the expectation, then get the following matrix differential equation with the initial condition: ("tr" is the trace of a matrix, (  ) =   ,  = 1, 2, . . ., ), the quadratic constraints are where  ) .
By the virtue of ( 17)-(18), constraints ( 14)-( 16) can be reformulated as ) . (24) Proof.The proof is mainly based on Definition 2. Apply Definition 2 to the functional constraints (19); the proofs for the first three results are the same to those in [8], and we only need the proof of (24) here: ) . ( are onto mapping when Proof.The proof of the onto mapping for (, ; Δ, Δ) has been given in [8]; we only need to prove that (; Δ) is a onto mapping when Δ varies.For a given  ∈   , the following equation has a solution: Because the coefficient matrix for quadratic terminal state constraints is full row rank (Assumption  1 ), equation exists a solution, which finishes the proof.
Definition 10 (the well-posedness).The LQ optimal control problem is to minimize the cost functional ( 0 , ) over  ∈  ad .Define the optimal value function as ( 0 ) = inf ∈ ad ( 0 , ).The LQ problem is called well posed if A well-posed problem is called attainable (with respect to  0 ) if there is a  * (⋅) control that achieves ( 0 ).
where  is a constant.
Lemma 15 (see [9]).If () and () are continuous in [ 1 ,  2 ] and We make use of NBV × [0, ] (nonnegative bounded variation functional on [0, ]) to express the matrix space with the element in [0, ].The space is a bounded variation right continuous function that function takes value 0 at the point at the point  = 0. Based on the Lagragian theorem and the Riesz representation theorem, we obtained a necessary condition for Problem 1.

Necessary Condition
Theorem 16 (necessary condition for indefinite stochastic LQ with quadratic constraints).Suppose the set defined by the terminal state constraint is not empty and the optimal control Problem 6 exists an optimal feedback control matrix  * ; then there must exist a symmetric matrix  ∈  × [0, ] and a vector  ∈   such that for all  ∈ [0, ],  and  satisfy the following equations: The second parts of (38) and (39) are from Riesz representation theorem.In general, we take that () = 0, and then (38) becomes It is obvious that there is no jump in interval [0, ) for function (), otherwise we can choose Δ that makes tr ∫  0 Δ be far more than the other parts in the equality.But () has jump at the point  = , and the height is . . .tr (   ()) ) .
Because the previous equalities (38) and ( 39) hold for all continuous functions Δ, for specified function which has continuous derivative and Δ() = 0, all the previous equalities also hold, and then Because () = 0, then According to Lemma 15, we have We take the integral by parts of the second part of (39), and then tr Based on Lemma 13, we have Change the endpoint conditions () = 0 with () =  1  1 + ⋅ ⋅ ⋅ +     ; that is, (36) holds.
Remark 18.The result in Theorem 12 is the same as that in [8] except for the terminal conditions.

Sufficient Condition.
We have pointed out that the necessary conditions ( 14)-( 16) and ( 35)-(37) are not sufficient for the existence and the uniqueness of the solution in Problem 6.In order to guarantee the uniqueness of  and (), the conditions must be strengthened to that is, the matrix  +    must be positive.

Conclusion
This paper studied a class of indefinite stochastic LQ control problems with quadratic terminal state constraints and mixed terminal state constraints.By means of the Lagrange multiplier theorem and Riesz representation theorem, this paper presented a necessary condition for indefinite stochastic LQ control problems with quadratic terminal state constraints and mixed terminal state constraints.The result shows that the necessary condition for quadratic terminal constraints is the same as for the linear terminal state constraints that is presented in [8] except for the terminal condition.This coincides with the reality.A sufficient condition also was presented for the existence and uniqueness of the optimal linear feedback control.Numerical example verified the main theorem in this paper.
() is continuously Frechet differentiable if, for a given  0 ∈ , ( 0 ; ℎ) is an onto mapping from  onto ; then  0 is called a regular point.Considering Problem 1 under the linear feedback optimal control, substituting () in Problem 1 with Definition 4 (Frechet derivative and continuously Frechet differentiable).Suppose that a transformation  defined on an open domain  ⊂  is Frechet differentiable on .The Frechet differential (; ℎ) =   ()ℎ for a fixed  ∈ , where   () is a bounded linear operator from  to , one calls   () the derivative of ().If the derivative   () is continuous on some open ball , then  is continuously Frechet differentiable on .Definition 5 (regular point of transformation).Let () be a transformation defined on a Banach space  with a range , which is also a Banach space.
Ẋ = ( + )  + ( + )  + ( + ) ( + )  , (14) Theorem 11 (Lagrange multiplier theorem).If the continuously Frechet differentiable real functional  defined on Banach space has a local extremum under the constraint () = 0 at the regular point  0 , () is a mapping from space  to Banach space , and then there exists an element  * 0 ∈  * such that the where the norm of () is the total variation on [, ].Conversely, every function of bounded variation on [, ] defines a bounded linear function on  in this way.