Mathematical Problems in Engineering

Volume 2013 (2013), Article ID 613159, 7 pages

http://dx.doi.org/10.1155/2013/613159

Research Article

Optimality Conditions for Optimal Control of Jump-Diffusion SDEs with Correlated Observations Noises

Hua Xiao

School of Mathematics and Statistics, Shandong University, Weihai 264209, China

Received 25 January 2013; Accepted 5 March 2013

Academic Editor: Guangchen Wang

Copyright © 2013 Hua Xiao. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

This paper is concerned with necessary and sufficient optimality conditions for optimal control of jump-diffusion stochastic differential equations. Compared with the existing literature, there are two distinguishing features: one is that the states are driven by Brownian motions and Poisson random measure; the other one is that the states and the observations are correlated. We derive a necessary and a sufficient conditions in the form of maximum principle when control domain is convex. A linear-quadratic example is worked out to illustrate the applications of the foregoing optimality conditions.

1. Introduction

The purpose of this paper is to establish maximum principle, also called necessary optimality conditions, for optimal control of jump-diffusion stochastic differential equations driven by Brownian motions and Poisson random measure where states and observations are correlated. There is rich literature on maximum principle for optimal control of stochastic differential equations (SDEs, for short). For example, Peng [1] proved a general stochastic maximum principle for optimal control of diffusion SDEs by introducing second-order variational equations; Tang and Li [2] derived a general necessary condition for optimal control of jump-diffusion SDEs; Pham [3] gave a survey on some recent aspects and developments in stochastic control of diffusion processes and discussed the main historical approaches and their modern exposition for studying stochastic control problems. In many situations, the states of the systems cannot be completely observed; however, some other processes related to the unobservable states can be observed. Then partially observable optimal control of SDEs also attracts much research attention. Such a subject has been discussed by many authors, such as Baras et al. [4], Bensoussan [5], Fleming [6], Li and Tang [7], Tang and Hou [8], Huang et al. [9], Wang and Wu [10, 11], Xiao [12], Zhou [13], and Hu and Øksendal [14].

In the real world, there usually exists certain correlated noises between the state and observation which is more general case than the case of independent noises. However, the literature mentioned above only deals with the case that states and observations are independent. To the author's best knowledge, there are only two papers about correlated observation noises. Tang [15] considered optimal control of diffusion SDEs and proved a general stochastic maximum principle; Xiao [16] obtained maximum principle for optimal control of Poisson point processes with correlated Gaussian white noisy observations. Up till now, there is no literature on optimal control of jump-diffusion SDEs with correlated observation noises. Here, we set out to study the optimal control problem of SDEs driven by Brownian motions and Poisson random measure in the case of correlated noisy observations. Due to the correlation between the state and the observation, there exists a weak solution rather than a strong solution to the state equation, which is different from the case of noncorrelated noisy observation. Meanwhile, the superfluous adjoint processes arise in the optimality condition, and we need some complicated matrices decomposition to eliminate them. We also make an effort to seek a suitable adjoint of the drift coefficient of the observation equation which plays an important role in defining the new Hamiltonian function, so that the improved necessary maximum principle is just the thing we want.

The rest of this paper is organized as follows. In Section 2, we formulate the optimal control problem of partially observable jump-diffusion SDEs with correlated observation noises. In Section 3, we obtain the partially observed stochastic maximum principle when the control domain is convex. Section 4 presents a linear-quadratic example to illustrate the applications of the theoretical results derived in Section 3. Some conclusions are given in Section 5.

2. Formulation of Problem

Throughout this paper, we assume that is a nonempty Borel subset of ,   is the Borel -algebra generated by , and is a -finite measure on . Let be a fixed real number. Let be a stationary -Poisson point process on with the characteristic measure . We denote by the counting measure or Poisson measure induced by and set satisfying and for every . In general, we call and the intensity and the compensated Poisson measure of , respectively (see Ikeda and Watanabe [17]). In addition, let be a complete probability space on which two mutually independent standard Brownian motions and are defined, valued in and , respectively, independent of . Let , , and be the natural filtration generated by , , and , respectively. We assume that

where denotes the totality of -null sets. For a matrix, we use superscripts to indicate (when necessary) the number of its columns or its rows or the position of its components, and precise meaning can be specified from the context; the range of the superscripts will not be explicitly stated unless there is a danger of confusion.

Let be a finite-dimensional space, and we denote by the space of -valued squared integrable -measurable random variables, by the space of -valued squared integrable -adapted processes, by the space of square integrable functions , by the space of -valued -predictable processes , such that . If , we write for . Unless otherwise stated, the notations and denote the above different meanings throughout the paper.

Let be a nonempty convex subsets of some Euclidean space. A control is a stochastic process . For notational simplicity, hereinafter, we will omit in random functions. We define an admissible control set by

Every element of is called an admissible control.

Introduce the mappings .

We make the following hypothesis.

(H1) , and are continuously differentiable with respect to . They are bounded by and their derivatives with respect to are uniformly bounded. and are uniformly bounded and continuously differentiable with respect to , whose derivatives with respect to are also uniformly bounded. and are continuously differentiable with respect to . There exists a constant such that .

The partially observable optimal control problem is stated as follows.

Consider the state

and the observation

where is a stochastic process depending on the control . Note that if the diffusion term in (3), then there exist the correlated noise between the state and observation. Substituting (4) into (3), we have

For each , there exists a unique strong solution to (5) (see Ikeda and Watanabe [17], which will be denoted by . We define

that is

and . Note that since is uniformly bounded, we actually have that is a martingale on , then the expression is to say that , that is, is absolutely continuously w.r.t. . Note that a.s., so we also have that . Hence the two measures and are equivalent. Furthermore, . Therefore is also a probability measure. From Girsanov's theorem, it follows that is a weak solution on to (3) and (4). Throughout this paper, the superscript symbol * means the transpose of certain vector or matrix.

Consider the cost functional

Here, denotes the expectation with respect to the probability space . Our partially observed optimal control problem is to minimize the cost functional (8) over , subject to (3) and (4).

Obviously, the cost functional (8) can be rewritten as

So the original optimization problem is equivalent to minimizing the cost functional (9) over , subject to (5) and (7).

If an admissible control minimizes the cost functional (if it does exist), then it is called optimal and the corresponding weak solution to (3) and (4) is called the optimal trajectory.

3. Maximum Principle

In this section, we shall establish a necessary and a sufficient maximum principles.

We note that minimizing the cost functional (9) over , subject to (5) and (7), is similar to a completely observable optimal control problem except for the difference of admissible control set. It is a heuristic method to derive the maximum principle for partially observable optimal control from the maximum principle for completely observable optimal control. We now specify this point.

Let be an optimal control and the corresponding optimal trajectory. Set

Equations (5) and (7) can be compressed into the following form:

The cost function (9) is rewritten as

Our partially observed optimal control problem becomes the following minimization problem: to minimize in (12) over , subject to (11). The present formulation of the partially observable optimal control problem is quite similar to a completely observed optimal control problem; the only difference lies in the admissible controls class . We can follow the same arguments in the case of full information to derive the following desired maximum principle (see Tang and Li [2] in the case of complete information and convex control domain).

Define the Hamiltonian as follows:

For simplicity, we introduce the notation

Let be the unique -adapted square integrable solution of the first-order adjoint equation

From the maximum principle in Tang and Li [2], we obtain the following necessary maximum principle for partially observable optimal control.

Lemma 1. Assume that the hypothesis (H1) holds. Let be an optimal control and the optimal trajectory to (3) and (4). Then

holds for .

We note that since the variable , which is regarded as the state variable for a moment, appears in the optimal control problem in a linear way, some adjoint processes are superfluous in the above maximum principle (16). Inspired by the method in Tang [15], we set about dispensing with these adjoint processes and reformulate the above maximum principle.

Decompose the matrices , and into blocks in the following manner:

Then we can check the following:

We introduce a new Hamiltonian as follows:

We set

For simplicity, we introduce the notation: , and similar notation will be made for other functions . We denote

by . In view of the above calculations, (15) can be decomposed into the following equation, for :

It is the position to rewrite the maximum condition (16). We can verify the following:

Then we obtain the following necessary maximum condition.

Theorem 2 (necessary maximum principle). Assume that the hypothesis (H1) holds. Let be an optimal control and the corresponding solution of (22). Then

is true for any .

Subsequently, we set out to derive the sufficient optimality conditions for the foregoing optimal control problem. We introduce the following assumption.

(H2) is convex in . Function is independent of variables and .

Inspired by Xiao [12] and Huang et al. [18], we have the following theorem.

Theorem 3 (sufficient maximum principle). Let (H1) and (H2) hold, be -adapted, and be an admissible control with the corresponding trajectory . Furthermore, one supposes that satisfies (22), the Hamiltonian is convex in , and

Then is an optimal control.

Proof. By virtue of the convexity property of and , we have for all ,

with

Applying Itô's formula to and , we deduce

Since the Hamiltonian is convex in , we have

Since the function is independent of variables and , we have . Substituting (28) and (29) into (26), it follows immediately that

In virtue of (25), we have

Substituting (31) into (30), since , we have

We draw the desired conclusion.

4. An LQ Example

To illustrate that the foregoing theories may find the interesting applications in practice, we work out an LQ example of partially observable optimal control of jump-diffusion system with correlated noisy observations. Firstly, by applying the necessary maximum principle, we find a candidate optimal control. Then by sufficient maximum principle, we verify that it is indeed an optimal control. Finally, by certain techniques of forward-backward stochastic differential equations filtering, we obtain an explicit expression of the optimal control.

Consider a partially observable 1-dimensional control system

with the observation

and the cost functional

Here, the coefficients and are bounded and deterministic. The set of admissible controls is defined by

denotes the expectation with respect to the probability space , and

We aim to find an explicitly optimal control to minimize the cost functional over , subject to (33) and (34).

Now we begin to seek the explicit optimal control by three steps.

First Step. Find candidate optimal controls.

We firstly write down the Hamiltonian function

where is the trajectory to (33) corresponding to the candidate optimal control . By Theorem 2, we find a unique candidate optimal control which satisfies the following expression:

where is the solution of the following equations under measure

Second Step. Verify in (39) is indeed optimal.

We can check that all conditions in Theorem 3 are satisfied, so in (39) is indeed optimal.

Third Step. Give explicit expression of optimal control.

Although we obtain its nominal form, the expression of optimal control in (39) is not quite explicit and satisfactory. From (33), we know that the state is dependent on the control . Since and are coupled at time and the control in (39) is dependent on the adjoint state , (33) and (40) compose mutually coupled forward-backward stochastic systems which makes it difficult to find their explicit solution. Next, we try to get a more explicit and observable expression than the one in (39) by certain filtering techniques.

We introduce the notation , for . Then the optimal control in (39) can be rewritten as

Obviously, from (41), if we deduce the optimal filter of , then we obtain the optimal control . Since is a deterministic function on , we know from (34) that . Then we have . Under measure , is a Brownian motion, but is not. In order to solve the explicit filter , from Xiong [19, Lemma  5.4], we can deduce the following group of filtering equations:

In terms of the monotonicity conditions introduced by Peng and Wu [20], (42) has a unique solution . Substituting and associated with into (35), we derive the optimal cost functional.

5. Conclusion Remark

This paper has investigated the optimal control problem of partially observable jump-diffusion SDEs. The most distinguishing feature is, compared with the existing literature, that the states and observations are correlated. By transforming the partial observation problem to a related problem with full information, we established a necessary and a sufficient optimality conditions. Under the framework of convex control domain, our maximum principle can cover Tang and Hou [8], Tang [15] and Xiao [12]) as particular cases, but does not establish the relations among the adjoint processes. In the future work, we shall introduce some adjoint vector fields to characterize the adjoint processes and derive some other formulations of partially observable stochastic maximum principle.

Acknowledgments

This research project was funded by the National Nature Science Foundation of China (11201263, 11071144, and 11101242), the Nature Science Foundation of Shandong Province (ZR2012AQ004, BS2011SF010), and Independent Innovation Foundation of Shandong University (IIFSDU), China.

References

S. Peng, “General stochastic maximum principle for optimal control problems,” SIAM Journal on Control and Optimization, vol. 28, no. 4, pp. 966–979, 1990.

S. Tang and X. Li, “Necessary conditions for optimal control of stochastic systems with random jumps,” SIAM Journal on Control and Optimization, vol. 32, no. 5, pp. 1447–1475, 1994.

H. Pham, “On some recent aspects of stochastic control and their applications,” Probability Surveys, pp. 506–549, 2005.

J. S. Baras, R. J. Elliott, and M. Kohlmann, “Partially observed stochastic minimum principle,” SIAM Journal on Control and Optimization, vol. 27, no. 6, pp. 1279–1292, 1989.

A. Bensoussan, “Maximum principle and dynamic programming approaches of the optimal control of partially observed diffusions,” Stochastics, vol. 9, no. 3, pp. 169–222, 1983.

W. Fleming, “Optimal control of partially observable diffusions,” SIAM Journal on Control and Optimization, vol. 6, pp. 194–214, 1968.

X. Li and S. Tang, “General necessary conditions for partially observed optimal stochastic controls,” Journal of Applied Probability, vol. 32, pp. 1118–1137, 1995.

S. Tang and S. H. Hou, “Optimal control of point processes with noisy observations: the maximum principle,” Applied Mathematics and Optimization, vol. 45, no. 2, pp. 185–212, 2002.

J. Huang, G. Wang, and Z. Wu, “Optimal premium policy of an insurance firm: full and partial information,” Insurance, vol. 47, no. 2, pp. 208–215, 2010.

G. Wang and Z. Wu, “Kalman-Bucy filtering equations of forward and backward stochastic systems and applications to recursive optimal control problems,” Journal of Mathematical Analysis and Applications, vol. 342, no. 2, pp. 1280–1296, 2008.

G. C. Wang and Z. Wu, “General maximum principles for partially observed risk-sensitive optimal control problems and applications to finance,” Journal of Optimization Theory and Applications, vol. 141, no. 3, pp. 677–700, 2009.

H. Xiao, “The maximum principle for partially observed optimal control of forward-backward stochastic systems with random jumps,” Journal of Systems Science and Complexity, vol. 24, no. 6, pp. 1083–1099, 2011.

X. Y. Zhou, “Sufficient conditions of optimality for stochastic systems with controllable diffusions,” IEEE Transactions on Automatic Control, vol. 41, no. 8, pp. 1176–1179, 1996.

Y. Hu and B. Øksendal, “Partial information linear quadratic control for jump diffusions,” SIAM Journal on Control and Optimization, vol. 47, no. 4, pp. 1744–1761, 2008.

S. Tang, “The maximum principle for partially observed optimal control of stochastic differential equations,” SIAM Journal on Control and Optimization, vol. 36, no. 5, pp. 1596–1617, 1998.

H. Xiao, “Maximum principle for optimal control of point processes with correlated noisy observations,” in Proceedinngs of the 30th Chinese Control Conference (CCC '11), pp. 1921–1924, Yantai, China, July 2011.

N. Ikeda and S. Watanabe, Stochastic Differential Equations and Diffusion Processes, North- Holland, New York, NY, USA; Kodansha, Tokyo, Japan, 1989.

J. Huang, X. Li, and G. Wang, “Maximum principles for a class of partial information risk-sensitive optimal controls,” IEEE Transactions on Automatic Control, vol. 55, no. 6, pp. 1438–1443, 2010.

J. Xiong, An Introduction to Stochastic Filtering Theory, Oxford University Press, London, UK, 2008.

S. Peng and Z. Wu, “Fully coupled forward-backward stochastic differential equations and applications to optimal control,” SIAM Journal on Control and Optimization, vol. 37, no. 3, pp. 825–843, 1999.