^{1}

This paper studies a discrete-time stochastic LQ problem over an infinite time horizon with state-and control-dependent noises, whereas the weighting matrices in the cost function are allowed to be indefinite. We mainly use semidefinite programming (SDP) and its duality to treat corresponding problems. Several relations among stability, SDP complementary duality, the existence of the solution to stochastic algebraic Riccati equation (SARE), and the optimality of LQ problem are established. We can test mean square stabilizability and solve SARE via SDP by LMIs method.

Stochastic linear quadratic (LQ) control problem was first studied by Wonham [

In this paper, we use SDP approach introduced in [

The organization of this paper is as follows. In Section

Consider the following discrete-time stochastic system:

We first give the following definitions.

The following system

System (

For system (

The LQ problem is called well-posed if

Stochastic algebraic Riccati equation (SARE) is a primary tool in solving LQ control problems. Associated with the above LQ problem, there is a discrete SARE:

A symmetric matrix

Throughout this paper, we assume that system (

The following definitions and lemmas will be used frequently in this paper.

For any matrix

Suppose that

Let matrices

Let matrices

For a symmetric matrix

Suppose that

Definition

Let

The primal problem (

The dual problem (

If both conditions hold, the optimal sets

The stabilization assumption of system (

System (

There are a matrix

There are a matrix

For any matrix

For any matrix

There exist matrices

Below, we will construct the relation between the stabilization and the dual SDP. First, we assume that the interior of the set

Consider the following SDP problem:

By the definition of SDP, we can get the dual problem of (

The dual problem of (

The objective of the primal problem can be rewritten as maximizing

This proof is simpler than the proof in [

The following theorem reveals that the stabilizability of discrete stochastic system can be also regarded as a dual concept of optimality. This result is a discrete edition of Theorem 6 in [

The system (

First, we prove the necessary condition. Assume that system (

Next, we prove the sufficient condition. Assume that the dual problem is strictly feasible; that is,

The following theorem will state the existence of the solution of the SARE (

The optimal set of (

Since system (

The following theorem shows that any optimal solution of the primal SDP results in a stabilizing control for LQ problem.

Let

Optimal dual variables

There is a unique optimal solution to (

The proof is similar to Theorem 9 in [

Assume that

The first assertion is Theorem 4 in [

Here we drop the assumption that the interior of

Theorem

The above results represent SARE (

The solvability margin

By Theorem

The solvability margin

Consider the system (

In order to test the mean square stabilizability of system (

Let

In this paper, we use the SDP approach to the study of discrete-time indefinite stochastic LQ control. It was shown that the mean square stabilization of system (

This work was supported by the National Natural Science Foundation of China (61174078), Specialized Research Fund for the Doctoral Program of Higher Education (20103718110006), and Key Project of Natural Science Foundation of Shandong Province (ZR2009GZ001).