Mathematical Problems in Engineering

Volume 2014 (2014), Article ID 278142, 9 pages

http://dx.doi.org/10.1155/2014/278142

Research Article

Discrete-Time Indefinite Stochastic Linear Quadratic Optimal Control with Second Moment Constraints

Weihai Zhang¹ and Guiling Li²

¹College of Electrical Engineering and Automation, Shandong University of Science and Technology, Qingdao 266590, China

²College of Mathematics and Systems Science, Shandong University of Science and Technology, Qingdao 266590, China

Received 1 April 2014; Accepted 29 April 2014; Published 18 May 2014

Academic Editor: Xuejun Xie

Copyright © 2014 Weihai Zhang and Guiling Li. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

This paper studies the discrete-time stochastic linear quadratic (LQ) problem with a second moment constraint on the terminal state, where the weighting matrices in the cost functional are allowed to be indefinite. By means of the matrix Lagrange theorem, a new class of generalized difference Riccati equations (GDREs) is introduced. It is shown that the well-posedness, and the attainability of the LQ problem and the solvability of the GDREs are equivalent to each other.

1. Introduction

LQ control, initiated by Kalman [1] and extended to stochastic systems by Wonham [2], is one of the most important classes of optimal control issues from both theory and application point of view; we refer the reader to [2–8] for representative work in this area. Different from the classical LQ in modern control theory, it was found in [9, 10] that a stochastic LQ problem with indefinite control weighting matrices can still be well-posed, which evoked a series of subsequent researches; see, for example, [11, 12].

It is well known that in practical engineering, the system state and control input are always subject to various constraints, so how to solve the constrained stochastic LQ issue is a more attractive topic; we refer the reader to [13–19]. Reference [14] presented a tractable approach for LQ controller design of the system with additive noise. Reference [16] was about the constrained LQ of deterministic systems with state equality constraints. Reference [13] studied the parametrization of the solutions of finite-horizon constrained LQ control. Reference [15] was devoted to a stochastic LQ optimal control and an application to portfolio selection, where the control variable is confined to a cone, and all the coefficients of the state equation are random processes. Reference [19] studied the indefinite stochastic LQ control problem of continuous-time Itô systems with a linear equality constraint on the terminal state and gave a necessary condition for the existence of an optimal controller. Reference [20] generalized the results of [19] to discrete-time stochastic systems.

In this paper, different from [19, 20] on the constraint conditions, we would like to deal with stochastic LQ control of discrete-time multiplicative noise systems with a second moment constraint and such constraints are often encountered in filtering design; see [21, 22]. By means of Lagrange theorem, we present a necessary condition for the existence of an optimal linear state feedback control with the second moment constraint on the terminal states. It is proved that the solvability of GDRE is necessary and sufficient for the existence of an optimal control under either of the state feedback case or of the open-loop forms. Moreover, we show that the well-posedness and the attainability of the constrained LQ problem, the feasibility of the LMI, and the solvability of the GDRE are equivalent to each other. The novel contribution of this paper is to consider a constrained discrete-time LQ optimal stochastic control, which includes some results of [23] as special cases. A new class of generalized difference Riccati equations (GDREs) is first introduced.

The remainder of the paper is organized as follows. Section 2 gives some definitions and preliminaries. In Section 3, the optimal state feedback control is studied using the matrix Lagrange theorem. We give a necessary and sufficient condition for the well-posedness of the constrained LQ control in Section 4. Section 5 shows the equivalence among the well-posedness and the attainability of the LQ problem, the feasibility of the LMI, and the solvability of the GDRE. The set of all optimal controls is determined. We conclude the paper in Section 6.

Throughout the paper, the following notations are adopted: denotes the transpose of . : is a positive definite (positive semidefinite) symmetric matrix. : the trace of a square matrix . : the space of all matrices. : the space of all symmetric matrices.

2. Problem Setting

Consider the following constrained discrete-time stochastic LQ control problem.

Problem 1. Consider

where the state , the control input , and the noise , ,

The process is a sequence of second-order stationary random variables defined on a complete probability space . Without loss of generality, we assume that

where is the Kronecker delta, . is a constant, , and are matrices having appropriate dimensions determined from context, and and are real symmetric indefinite matrices. is a given deterministic vector.

Definition 2. Problem 1 is called well-posed, if ,

Definition 3. Problem 1 is called attainable, if , there exists a sequence , such that . In this case, is called an optimal control sequence.

Now, let us consider a mathematical programming (MP) problem in a matrix space:

Definition 4. Let be a point satisfying

and then is said to be a constraint regular point if the gradient vectors , are linearly independent.

Lemma 5 (Lagrange theorem [24]). Assume that the functions , , are twice continuously differentiable. If a regular point is also a relative minimum point for the original MP, then there exists a vector such that

where the Lagrangian function .

3. A Necessary Condition for State Feedback Control

In this section, by the matrix Lagrange theorem, we present a necessary condition for Problem 1 based on a new type of GDREs.

Let . Through a simple calculation, the following deterministic optimal control Problem 6 is equivalent to the original Problem 1 under the state feedback for .

Problem 6. Consider

with

Remark 7. If Problem 1 has a linear feedback optimal control solution , , then , , are the optimal solution of Problem 6.

Theorem 8. If Problem 1 is attainable by , and the regular point is a locally optimal solution of Problem 6, then there exist symmetric matrices , and solving the following GDRE:

where is the Moore-Penrose generalized inverse of . Moreover,

with , , being any given real matrices:

To prove Theorem 8, we mainly use Lemma 5 to Problem 6 together with the following lemma to obtain GDRE (10) and then apply the technique of completing squares to show (12).

Lemma 9 (see [12]). Let be given matrices with appropriate sizes; then the matrix equation

has a solution if and only if

Moreover, any solution to can be represented by

where is any matrix with appropriate size.

Proof. According to Remark 7, is also the optimal solution of Problem 6. Problem 6 is a typical MP problem about and as follows:

where

Let matrices , , be the Lagrangian multipliers of

and let be the Lagrangian multiplier of ; then the Lagrangian function

According to the the matrix Lagrange theorem, we obtain

Based on the partial rule of gradient matrices, (20) can be transformed into

Let

Then we obtain

Applying Lemma 9, we have

Equation (21) yields

Substituting into (26), it follows

Without loss of generality, we can assume that is symmetric. Otherwise, we can take . The objective functional

A completion of square implies

We assert that must satisfy

If it is not so, there is for with a negative eigenvalue . Denote the unitary eigenvector with respect to by . Let be an arbitrary scalar; we construct a control sequence as follows:

The associated cost functional becomes

Let ; then , which contradicts the attainability of Problem 1. So (30) holds.

In view of (29) and (30), (11) and (12) are easily derived. The proof is completed.

Remark 10. In Theorem 8, in order to apply matrix Lagrange theorem, we assume the optimal solution is a regular point. Generally speaking, for a given LQ control, it is easy to examine the regular condition.

Below, we present a numerical example to illustrate the effectiveness of Theorem 8.

Example 11. In Problem 1, we set

The state and control weighting matrices are as

By the relationship between Problems 1 and 6, we know

Applying Theorem 8, we obtain

Stage 2. Consider

Stage 1. Consider

The optimal cost value of Problem 1 is

We are able to test the regular condition of as follows. In Problem 6,

which is linear about and quadratic about , while is linear about . By simple calculations, , , and are all nonzero vectors and hence are linearly independent.

4. Well-Posedness

In this section, we first establish the link between the well-posedness of Problem 1 and the feasibility of some LMIs and then prove that the solvability of GDRE (10) is not only necessary but also sufficient to the well-posedness of Problem 1. Moreover, the well-posedness and the attainability of Problem 1, the feasibility of some LMIs, and the solvability of GDRE (10) are equivalent to each other.

Theorem 12. Problem 1 is well-posed if there exist symmetric matrices , and solving the following LMIs:

where

Proof. Note that

By (41), it is easy to deduce that the cost functional is bounded from below by

Hence, Problem 1 is well-posed.

Remark 13. Theorem 12 tells us that any symmetric matrices , and satisfying LMIs (41)-(42) provide a lower bound

for the cost function. In what follows, we will show that this lower bound is an exact optimal cost value if and solve GDRE (10).

We have shown that if the LMIs (41)-(42) are satisfied, then the constrained LQ Problem 1 is well-posed. Below, we further show some other equivalent conditions.

Lemma 14 (extended Schur’s lemma [25]). Let the matrices , , be given with appropriate sizes. Then, the following three conditions are equivalent:

Theorem 15. Problem 1 is well-posed if and only if there exist symmetric matrices , and satisfying GDRE (10). Furthermore, the optimal cost is

A key to prove Theorem 15 is the necessity part, where the stochastic optimization principle is used.

Proof. Necessity. For , define

By the stochastic optimization principle, when is finite, then so is for any . Since Problem 1 is assumed to be well-posed at , is finite at any stage . Now let us start with , and let , and we have

Since is finite, using Lemma 4.3 of [23], there exists a symmetric matrix such that

The obtained solution sequence of symmetric matrices , , and to GDRE (10) satisfy

Then by the stochastic optimality principle, the following holds:

Lemma 4.3 of [23] provides necessary and sufficient conditions for the finiteness of :

Moreover,

The above proves the necessity part via mathematical induction.

Sufficiency. From the proof of Theorem 8, if GDRE (10) admits a solution and , Problem 1 is not only well-posed, but also attainable. The proof of this theorem is complete.

5. Other Equivalent Conditions

In this section, we present some other equivalent conditions for Problem 1.

Theorem 16. For the constrained LQ Problem 1, the following are equivalent:

(i)Problem 1 is well-posed.(ii)Problem 1 is attainable.(iii)The LMIs (41)-(42) are feasible.(iv)The GDRE (10) is solvable.

Furthermore, when any one of the above conditions is satisfied, Problem 1 is attainable by

where , are solutions to the GDRE (10).

Proof. Applying Theorems 12–15, . is shown by Theorem 8. The rest is to prove and (56). Let , solve the GDRE (10). In view of

a completion of squares yields

which shows

Finally, we present a general expression for the optimal control set based on the solution to GDRE (10).

Theorem 17. Assume that the GDRE (10) admits a solution. Then the set of all optimal controls is determined by

where and are arbitrary random variables defined on the probability space . Moreover, the optimal cost value is uniquely given by

where , and are the solution to the GDRE (10).

Proof. This theorem can be proved by repeating the same procedure as in Theorem 5.1 of [23].

6. Conclusion

In this paper, we have investigated a class of indefinite stochastic LQ control problems with second moment constraints on the terminal state. By the matrix Lagrange theorem, we have established a new GDRE (10) associated with the constrained optimization Problem 1. In addition, by introducing LMIs (41)-(42), we show that the well-posedness and the attainability of Problem 1, the feasibility of the LMIs (41)-(42), and the solvability of GDRE (10) are equivalent to each other.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This work is supported by the National Natural Science Foundation of China (no. 61174078), the Research Fund for the Taishan Scholar Project of Shandong Province of China and SDUST Research Fund (no. 2011KYTD105), and State Key Laboratory of Alternate Electrical Power System with Renewable Energy Sources (Grant no. LAPS13018).

References

R. E. Kalman, “Contributions to the theory of optimal control,” Boletín de la Sociedad Matemática Mexicana, vol. 5, no. 2, pp. 102–119, 1960.

W. M. Wonham, “On a matrix Riccati equation of stochastic control,” SIAM Journal on Control and Optimization, vol. 6, no. 4, pp. 681–697, 1968.

C. E. de Souza and M. D. Fragoso, “On the existence of maximal solution for generalized algebraic Riccati equations arising in stochastic control,” Systems and Control Letters, vol. 14, no. 3, pp. 233–239, 1990.

B. S. Chen and W. Zhang, “Stochastic control with state-dependent noise,” IEEE Transactions on Automatic Control, vol. 49, no. 1, pp. 45–57, 2004.

W. Zhang and B. S. Chen, “On stabilizability and exact observability of stochastic systems with their applications,” Automatica, vol. 40, no. 1, pp. 87–94, 2004.

B. D. O. Anderson and J. B. Moore, Optimal Control Linear Quadratic Mathods, Prentice-Hall, New York, NY, USA, 1989.

F. L. Lewis, Optimal Control, John Wiley and Sons, New York, NY, USA, 1986.

Y. Huang, W. Zhang, and H. Zhang, “Infinite horizon linear quadratic optimal control for discrete-time stochastic systems,” Asian Journal of Control, vol. 10, no. 5, pp. 608–615, 2008.

S. Chen, X. J. Li, and X. Y. Zhou, “Stochastic linear quadratic regulators with indefinite control weight costs,” SIAM Journal on Control and Optimization, vol. 36, no. 5, pp. 1685–1702, 1998.

M. A. Rami and X. Y. Zhou, “Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls,” IEEE Transactions on Automatic Control, vol. 45, no. 6, pp. 1131–1143, 2000.

D. D. Yao, S. Zhang, and X. Y. Zhou, “Stochastic linear-quadratic control via semidefinite programming,” SIAM Journal on Control and Optimization, vol. 40, no. 3, pp. 801–823, 2001.

M. Ait Rami, J. B. Moore, and X. Y. Zhou, “Indefinite stochastic linear quadratic control and generalized differential Riccati equation,” SIAM Journal on Control and Optimization, vol. 40, no. 4, pp. 1296–1311, 2001.

A. Ferrante, G. Marro, and L. Ntogramatzidis, “A parametrization of the solutions of the finite-horizon LQ problem with general cost and boundary conditions,” Automatica, vol. 41, no. 8, pp. 1359–1366, 2005.

D. Bertsimas and D. B. Brown, “Constrained stochastic LQC: a tractable approach,” IEEE Transactions on Automatic Control, vol. 52, no. 10, pp. 1826–1841, 2007.

Y. Hu and X. Y. Zhou, “Constrained stochastic LQ control with random coefficients, and application to portfolio selection,” SIAM Journal on Control and Optimization, vol. 44, no. 2, pp. 444–466, 2005.

S. Ko and R. R. Bitmead, “Optimal control for linear systems with state equality constraints,” Automatica, vol. 43, no. 9, pp. 1573–1582, 2007.

A. Kojima and M. Morari, “LQ control for constrained continuous-time systems,” Automatica, vol. 40, no. 7, pp. 1143–1155, 2004.

Z. Zhou and C. Randy, “An algorithm for state constrained stochastic linear-quadratic control,” in Proceedings of the American Control Conference, pp. 1476–1481, San Francisco, Calif, USA, June 2011.

Y. L. Huang and W. H. Zhang, “Study on stochastic linear quadratic optimal control with constraint,” Acta Automatica Sinica, vol. 32, no. 2, pp. 246–254, 2006 (Chinese).

G. Li and W. Zhang, “Discrete-time indefinite stochastic linear quadratic optimal control with equality constraints,” in Proceeding of the Chinese Control and Decision Conference, pp. 4999–5004, Guiyang, China, May 2013.

H. Dong, Z. Wang, D. W. C. Ho, and H. Gao, “Variance-constrained filtering for a class of nonlinear time-varying systems with multiple missing measurements: the finite-horizon case,” IEEE Transactions on Signal Processing, vol. 58, no. 5, pp. 2534–2543, 2010.

L. Ma, Y. Bo, Y. Zhou, and Z. Guo, “Error variance-constrained filtering for a class of nonlinear stochastic systems with degraded measurements: the finite horizon case,” International Journal of Systems Science, vol. 43, no. 12, pp. 2361–2372, 2012.

M. Ait Rami, X. Chen, and X. Y. Zhou, “Discrete-time indefinite LQ control with state and control dependent noises,” Journal of Global Optimization, vol. 23, no. 3-4, pp. 245–265, 2002.

D. P. Bertsekas, Constrained Optimization and Lagrange Multiplier Methods, Athena Scientific, Belmont, Mass, USA, 1996.

A. Albert, “Conditions for positive and nonnegative definiteness in terms of pseudoinverses,” SIAM Journal on Applied Mathematics, vol. 17, no. 2, pp. 434–440, 1969.