Mathematical Problems in Engineering
Volume 2015 (2015), Article ID 892304, 13 pages
http://dx.doi.org/10.1155/2015/892304

Research Article

Maximum Principle for Optimal Control Problems of Forward-Backward Regime-Switching Systems Involving Impulse Controls

Shujun Wang and Zhen Wu

School of Mathematics, Shandong University, Jinan, Shandong 250100, China

Received 15 April 2014; Accepted 28 August 2014

Academic Editor: Guangchen Wang

Copyright © 2015 Shujun Wang and Zhen Wu. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

This paper is concerned with optimal control problems of forward-backward Markovian regime-switching systems involving impulse controls. Here the Markov chains are continuous-time and finite-state. We derive the stochastic maximum principle for this kind of systems. Besides the Markov chains, the most distinguishing features of our problem are that the control variables consist of regular and impulsive controls, and that the domain of regular control is not necessarily convex. We obtain the necessary and sufficient conditions for optimal controls. Thereafter, we apply the theoretical results to a financial problem and get the optimal consumption strategies.

1. Introduction

Maximum principle was first formulated by Pontryagin et al.’s group [1] in the 1950s and 1960s, which focused on the deterministic control system to maximize the corresponding Hamiltonian instead of the optimization problem. Bismut [2] introduced the linear backward stochastic differential equations (BSDEs) as the adjoint equations, which played a role of milestone in the development of this theory. The general stochastic maximum principle was obtained by Peng in [3] by introducing the second order adjoint equations. Pardoux and Peng also proved the existence and uniqueness of solution for nonlinear BSDEs in [4], which has been extensively used in stochastic control and mathematical finance. Independently, Duffie and Epstein introduced BSDEs under economic background, and in [5] they presented a stochastic recursive utility which was a generalization of the standard additive utility with the instantaneous utility depending not only on the instantaneous consumption rate but also on the future utility. Then El Karoui et al. gave the formulation of recursive utilities from the BSDE point of view. As found by [6], the recursive utility process can be regarded as a solution of BSDE. Peng [7] first introduced the stochastic maximum principle for optimal control problems of forward-backward control system as the control domain is convex. Since BSDEs and forward-backward stochastic differential equations (FBSDEs) are involved in a broad range of applications in mathematical finance, economics, and so on, it is natural to study the control problems involving FBSDEs. To establish the necessary optimality conditions, Pontryagin maximum principle is one fundamental research direction for optimal control problems. Rich literature for stochastic maximum principle has been obtained; see [8–12] and the references therein. Recently, Wu [13] established the general maximum principle for optimal controls of forward-backward stochastic systems in which the control domains were nonconvex and forward diffusion coefficients explicitly depended on control variables.

The applications of regime-switching models in finance and stochastic control also have been researched in recent years. Compared to the traditional system based on the diffusion processes, it is more meaningful from the empirical point of view. Specifically, it modulates the system with a continuous-time finite-state Markov chain with each state representing a regime of the system or a level of economic indicator. Based on the switching diffusion model, much work has been done in the fields of option pricing, portfolio management, risk management, and so on. In [14], Crépey focused on the pricing equations in finance. Crépey and Matoussi [15] investigated the reflected BSDEs with Markov chains. For the controlled problem with regime-switching model, Donnelly studied the sufficient maximum principle in [16]. Using the results about BSDEs with Markov chains in [14, 15], Tao and Wu [17] derived the maximum principle for the forward-backward regime-switching model. Moreover, in [18] the weak convergence of BSDEs with regime switching was studied. For more results of Markov chains, readers can refer to the references therein.

In addition, stochastic impulse control problems have received considerable research attention due to their wide applications in portfolio optimization problems with transaction costs (see [19, 20]) and optimal strategy of exchange rates between different currencies [21, 22]. Korn [23] also investigated some applications of impulse control in mathematical finance. For a comprehensive survey of theory of impulse controls, one is referred to [24]. Wu and Zhang [25] first studied stochastic optimal control problems of forward-backward systems involving impulse controls, in which they assumed the domain of the regular controls was convex and obtained both the maximum principle and sufficient optimality conditions. Later on, in [26] they considered the forward-backward system in which the domain of regular controls was not necessarily convex and the control variable did not enter the diffusion coefficient.

In this paper, we consider a stochastic control system, in which the control system is described by a forward-backward stochastic differential equation, all the coefficients contain Markov chains, and the control variables consist of regular and impulsive parts. This case is more complicated than [17, 25, 26]. We obtain the stochastic maximum principle by using spike variation on the regular control and convex perturbation on the impulsive one. Applying the maximum principle to a financial investment-consumption model, we also get the optimal consumption processes and analyze the effects on consumption by various economic factors.

The rest of this paper is organized as follows. In Section 2, we give preliminaries and the formulation of our problems. A necessary condition in the form of maximum principle is established in Section 3. Section 4 aims to investigate sufficient optimality conditions. An example in finance is studied in Section 5 to illustrate the applications of our theoretical results and some figures are presented to give more explanations. In the end, Section 6 concludes the novelty of this paper.

2. Preliminaries and Problem Formulation

Let be a complete filtered probability space equipped with a natural filtration generated by , , where is a -dimensional standard Brownian motion defined on the space, is a finite-state Markov chain with the state space given by , and is a fixed time horizon. The transition intensities are for with nonnegative and bounded. . For , denote by the set of -dimensional adapted processes such that and denote by the set of -dimensional adapted processes such that .

Define as the integer-valued random measure on which counts the jumps from to state between time 0 and . The compensator of is , which means is a martingale (compensated measure). Then the canonical special semimartingale representation for is given by

Define . Denote by the set of measurable functions from to endowed with the topology of convergence in measure and the norm of ; denote by the space of -measurable functions such that .

Let be a nonempty subset of and nonempty convex subset of . Let be a given sequence of increasing -stopping times such that as . Denote by the class of right continuous processes such that each is an -measurable random variable. It’s worth noting that, the assumption implies that at most finitely many impulses may occur on . Denote by the class of adapted processes such that and denote by the class of -valued impulse processes such that . is called the admissible control set. For notational simplicity, in what follows we focus on the case where all processes are 1-dimensional.

Now we consider the forward regime-switching systems modulated by continuous-time, finite-state Markov chains involving impulse controls. Let , , and be measurable mappings. Given and , the system is formulated by

The following result is easily obtained.

Proposition 1. Assume that are Lipschitz with respect to , , , , and is a continuous function. Then SDE (2) admits a unique solution .

Given and , consider the following backward regime-switching system modulated by Markov chains involving impulse controls:

where and are measurable mappings and is a measurable function such that .

Proposition 2. Assume that is Lipschitz with respect to , , , and is a continuous function. Then BSDE (3) admits a unique solution .

Proof. Define and , . It is easy to check that

Since is uniformly bounded, we have

Here are positive constants. Then is Lipschitz with respect to . We also get that and . Hence, the following BSDE

admits a unique solution (see [15, 18] for details). Now define , , and . Then it is easy to check that solves BSDE (3).
Let and be two solutions of (3). Applying Itô’s formula to and combining Gronwall’s inequality, we get the uniqueness of solution.

Now, we consider the following stochastic control system:

where , , , and are deterministic measurable functions and , are continuous functions. In what follows will be written as for short. The objective is to maximize, over class , the cost functional

where , , and are deterministic measurable functions. A control which solves this problem is called an optimal control.

In what follows, we make the following assumptions.

(H1), , , , , , and are continuous and continuously differentiable with respect to . have linear growth with respect to . is continuous and continuously differentiable with respect to .(H2)The derivatives of , , , and are bounded.(H3)The derivatives of , and are bounded by , , , and , respectively. Moreover, for any .

From Propositions 1 and 2, it follows that, under (H1)–(H3), FBSDE (7) admits a unique solution for any .

3. Stochastic Maximum Principle

In this section, we will derive the stochastic maximum principle for optimal control problem (7) and (8). We give the necessary conditions for optimal controls.

Let and be an optimal control of this stochastic control problem and let be the corresponding trajectory. Now, we introduce the spike variation with respect to as follows:

where is an arbitrarily fixed time, is a sufficiently small constant, and is an arbitrary -valued -measurable random variable such that . Let be such that . For the reason that domain is convex, we can check that , , is also an element of . Let be the trajectory corresponding to . For convenience, we denote , for , where , .

Introduce the following FBSDE which is called the variational equation:

Obviously, this FBSDE admits a unique solution .

We have the following lemma. In what follows, we denote by a positive constant which can be different from line to line.

Lemma 3. Consider

Proof. By the boundedness of and using Hölder’s inequality, we have

. Noting the definition of , we get

Here we apply Hölder’s inequality for , , and the growth condition of in (H1). Since is bounded on , then (11) is obtained by applying Gronwall’s inequality.
By the result of Section  5 in [6] and noting that the predictable covariation of is

we obtain

On the one hand, since is bounded, by (11), we have

On the other hand, since is bounded, using the basic inequality and (11), we have

From the growth condition of in (H1) and the same technique as above, it follows that

Besides, is bounded on ; then (12) is obtained. The proof is complete.

Denote , , , and , and then we have the following.

Lemma 4. Consider

where as .

Proof. It is easy to check that satisfies

where

Then we have

. Since , by the boundedness of , we have . Further we get

On the other hand, since , we have

where

Since is bounded, by Lemma 3 we get

For , by Hölder’s inequality, Lemma 3, and the dominated convergence theorem, it follows that

Then we get

and obtain

In the same way, we have

From (24), (31), and (32) it follows that

Finally, applying Gronwall’s inequality implies (20).

To get estimate (21), for simplicity, we introduce

It is easy to check that satisfies

where

Similar to the proof above, we have

Then for BSDE (35), by the estimates of BSDEs, we obtain

Applying Hölder’s inequality, Cauchy-Schwartz inequality, the dominated convergence theorem, Lemma 3, and (20) and noting the boundedness of , we obtain (21).

Now, we are ready to state the variational inequality.

Lemma 5. The following variational inequality holds:

Proof. From the optimality of , we have

By Lemmas 3 and 4, we have

Similarly, we obtain

Next, we aim to get the first term of (39). For convenience, we introduce two notations as follows:

Applying the same technique to the proof of Lemma 4, we obtain

Hence

Thus, variational inequality (39) follows from (41)–(45).
Let us introduce the following adjoint equations:

where for . It is easy to check that SDE (46) admits a unique solution . Besides, the generator of BSDE (47) does not contain . Therefore, the Lipschitz condition is satisfied obviously. Hence (47) admits a unique solution . Now we establish the stochastic maximum principle.

Theorem 6. Let assumptions (H1)–(H3) hold. Suppose is an optimal control, is the corresponding trajectory, and is the solution of adjoint equations (46) and (47). Then, , it holds that

where is the Hamiltonian defined by

where .

Proof. Applying Itô’s formula to and combining with Lemma 5, we obtain

where such that . Then it follows that

Letting , we obtain

By choosing we get (49). Setting , then for any we have

Let for and . Obviously and . Then it follows that for any

which implies

The proof is complete.

4. Sufficient Optimality Conditions

In this section, we add additional assumptions to obtain the sufficient conditions for optimal controls. Let us introduce the following.

(H4)The control domain is a convex body in . The measurable functions , and are locally Lipschitz with respect to , and their partial derivatives are continuous with respect to .

Theorem 7. Let (H1)–(H4) hold. Suppose that the functions , , , and are concave and is the solution of adjoint equations (46) and (47) corresponding to control . Moreover, assume that is of the special form , , where is a deterministic measurable function and . Then is an optimal control if it satisfies (48) and (49).

Proof. Let be the trajectory corresponding to . By the concavity of and , we derive

Define

Applying Itô’s formula to and noting , we obtain

By (48) and Lemma 2.3 of Chapter  3 in [27], we have

By Lemma 2.4 of Chapter  3 in [27], we further conclude that

Finally, by the concavity of and (49), we obtain , . Thus, it follows that . We complete the proof.

5. Application in Finance

This section is devoted to studying an investment and consumption model under the stochastic recursive utility arising from financial markets, which naturally motivates the study of the problem (7) and (8).

5.1. An Example in Finance

In a financial market, suppose there are two kinds of securities which can be invested: a bond, whose price is given by

and a stock, whose price is

Here, is the standard Brownian motion and , , and are bounded deterministic functions. For the sake of rationality, we assume , . Here, stands for a positive constant, which ensures that is nondegenerate. In reality, in order to get stable profit and avoid risk of bankruptcy, many small companies and individual investors usually make a plan at the beginning of a year or a period, in which the weight invested in stock was fixed. Denote by the weight invested in stock which is called the portfolio strategy. It means no matter how much the wealth is, the portfolio strategy is fixed, which is a bounded deterministic function with respect to . Then the wealth dynamics are given as

where , , and . Here, is a continuous consumption process, is a piecewise consumption process, and is a weight factor. Not only in the mode of continuous consumption, but also in reality society, one consumes piecewise. Hence our setting of consumption process is practical.

Besides, if the macroeconomic conditions are also taken into account in this model, above model has obvious imperfections because it lacks the flexibility to describe the changing stochastically of investment environment. One can modulate the uncertainty of the economic situation by a continuous-time finite-state Markov chain. Then the wealth is formulated by a switching process as

Let be a nonempty subset of and a nonempty convex subset of . Suppose is the natural filtration generated by the Brownian motion and the Markov chains, is an -progressively measurable process satisfying

is a fixed sequence of increasing -stopping times, and each is an -measurable random variable satisfying

We consider the following stochastic recursive utility, which is described by a BSDE with the Markov chain :

where , . The recursive utility is meaningful in economics and theory. Details can be found in Duffie and Epstein [5] and El Karoui et al. [6].

Define the associated utility functional as

where , , and are positive constants, is a discount factor, and is also called Arrow-Pratt index of risk aversion (see, e.g., Karatzas and Shreve [28]). To get the explicit solution, we also assume . The first and second terms in (69) measure the total utility from and , while the third term characterizes the initial reserve . It is natural to desire to maximize the expected utility functional representing cumulative consumption and the recursive utility , which means to find satisfying (66) and (67), respectively, to maximize in (69).

We solve the problem by the maximum principle derived in Section 3. The Hamiltonian corresponding to this model is

where is the solution of the following adjoint equations:

From (71) it is easy to obtain that

To solve (72), we introduce the dual process

Actually, (74) is solved by

Applying Itô’s formula to and taking conditional expectation with respect to , we obtain

Note that ; then we have . Thus, by Theorem 6 we get the optimal consumption processes for the regime-switching investment-consumption problem (65)–(69) as follows:

where is given by (73) and (76), respectively.

5.2. Numerical Simulation

In this part, we calculate the optimal consumption functions explicitly according to (71)–(77) in the case that all coefficients are constants and discuss the relationship between consumption and some financial parameters, which can further illustrate our results obtained in this paper. We only consider the optimal regular consumption process and in this case the Markov chain has two states . Here will not change from to . Further we fix and year throughout this part.

5.2.1. The Relationship between and

As , we set

From Figure 1, we find that the higher the risk-free interest rate is, the lower the optimal consumption is. It coincides with the financial behaviors in reality. As the risk-free interest rate grows higher, the investors can gain more profits via deposit. Consequently, the desire of consumption is declined.

Figure 1: The relationship between and as .

As , we set

Figure 2 shows the influence of risk-free interest rate on the optimal consumption function as . Same as Figure 1, when the risk-free interest rate gets higher, the optimal consumption becomes smaller. From Figures 1 and 2, we also find that under different strategies of government’s macrocontrol (different ), the optimal consumption has different values and changes trends with respect to , even for the same risk-free interest rate . It is natural because affects some parameters in this model such as , , , , and .

Figure 2: The relationship between and as .

5.2.2. The Relationship between and

The following two figures show the relationships between the optimal consumption function and appreciation rate of stock. First, for , we fix

From Figure 3, we can see that the higher the appreciation rate of stock is, the lower the optimal consumption is. It is also reasonable since a higher appreciation rate of stock inspires investors to put more money into stock market and thereby reduce the consumption. For , we fix

Figure 3: The relationship between and as .

Figure 4 also presents the same influence of appreciation rate on the optimal consumption function as . In addition, Figures 3 and 4 enhance us to understand that the optimal consumption has different values and changes trends with respect to for the same appreciation rate by considering different strategies of government’s macrocontrol.

Figure 4: The relationship between and as .

Based on Figures 1–4, we analyze the relationships between the optimal consumption function and the risk-free interest rate, the appreciation rate of stock, and the government’s macrocontrol, which are quite important and applicable in financial problems.

6. Conclusion

In this paper, we consider the optimal control problem of forward-backward Markovian regime-switching systems involving impulse controls. The control system is described by FBSDEs involving impulse controls and modulated by continuous-time, finite-state Markov chains. Based on both spike and convex variation techniques, we establish the maximum principle and sufficient optimality conditions for optimal controls. Here, the regular control does not enter in the diffusion term of the forward system. In the future, we may focus on the cases that the diffusion coefficient contains controls, fully coupled forward-backward Markovian regime-switching system involving impulse controls, and game problems in this framework. It is worth pointing out that if the domain of regular control is not convex and the control enters in the forward diffusion coefficient, it will be more complicated and bring some difficulties immediately by applying spike variation. Based on the methods and results of [13], we hope to further research for such kind of control problems and investigate more applications in reality.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This work is supported by the National Natural Science Foundation of China under Grants (11221061 and 61174092), 111 Project (B12023), the Natural Science Fund for Distinguished Young Scholars of China (11125102), and the Science and Technology Project of Shandong Province (2013GRC32201).

References

L. Pontryagin, V. Boltyanskti, R. Gamkrelidze, and E. Mischenko, The Mathematical Theory of Optimal Control Processes, John Wiley & Sons, New York, NY, USA, 1962.

J. Bismut, “An introductory approach to duality in optimal stochastic control,” SIAM Review, vol. 20, no. 1, pp. 62–78, 1978.

S. G. Peng, “A general stochastic maximum principle for optimal control problems,” SIAM Journal on Control and Optimization, vol. 28, no. 4, pp. 966–979, 1990.

E. Pardoux and S. Peng, “Adapted solution of a backward stochastic differential equation,” Systems and Control Letters, vol. 14, no. 1, pp. 55–61, 1990.

D. Duffie and L. G. Epstein, “Stochastic differential utility,” Econometrica, vol. 60, no. 2, pp. 353–394, 1992.

N. El Karoui, S. Peng, and M. C. Quenez, “Backward stochastic differential equations in finance,” Mathematical Finance, vol. 7, no. 1, pp. 1–71, 1997.

S. Peng, “Backward stochastic differential equations and applications to optimal control,” Applied Mathematics and Optimization, vol. 27, no. 2, pp. 125–144, 1993.

W. S. Xu, “Stochastic maximum principle for optimal control problem of forward and backward system,” Journal of the Australian Mathematical Society B, vol. 37, no. 2, pp. 172–185, 1995.

Z. Wu, “Maximum principle for optimal control problem of fully coupled forward-backward stochastic systems,” Systems Science and Mathematical Sciences, vol. 11, no. 3, pp. 249–259, 1998.

J. Yong, “Optimality variational principle for controlled forward-backward stochastic differential equations with mixed initial-terminal conditions,” SIAM Journal on Control and Optimization, vol. 48, no. 6, pp. 4119–4156, 2010.

G. Wang and Z. Wu, “The maximum principles for stochastic recursive optimal control problems under partial information,” IEEE Transactions on Automatic Control, vol. 54, no. 6, pp. 1230–1242, 2009.

G. Wang and Z. Yu, “A Pontryagin's maximum principle for non-zero sum differential games of BSDEs with applications,” IEEE Transactions on Automatic Control, vol. 55, no. 7, pp. 1742–1747, 2010.

Z. Wu, “A general maximum principle for optimal control of forward-backward stochastic systems,” Automatica, vol. 49, no. 5, pp. 1473–1480, 2013.

S. Crépey, “About the pricing equations in finance,” in Paris-Princeton Lectures on Mathematical Finance 2010, vol. 2003 of Lecture Notes in Mathematics, pp. 63–203, Springer, Berlin, Germany, 2011.

S. Crépey and A. Matoussi, “Reflected and doubly reflected BSDEs with jumps: a priori estimates and comparison,” The Annals of Applied Probability, vol. 18, no. 5, pp. 2041–2069, 2008.

C. Donnelly, “Sufficient stochastic maximum principle in a regime-switching diffusion model,” Applied Mathematics and Optimization, vol. 64, no. 2, pp. 155–169, 2011.

R. Tao and Z. Wu, “Maximum principle for optimal control problems of forward-backward regime-switching system and applications,” Systems & Control Letters, vol. 61, no. 9, pp. 911–917, 2012.

R. Tao, Z. Wu, and Q. Zhang, “BSDEs with regime switching: weak convergence and applications,” Journal of Mathematical Analysis and Applications, vol. 407, no. 1, pp. 97–111, 2013.

M. H. A. Davis and A. R. Norman, “Portfolio selection with transaction costs,” Mathematics of Operations Research, vol. 15, no. 4, pp. 676–713, 1990.

B. Øksendal and A. Sulem, “Optimal consumption and portfolio with both fixed and proportional transaction costs,” SIAM Journal on Control and Optimization, vol. 40, no. 6, pp. 1765–1790, 2002.

A. Cadenillas and F. Zapatero, “Classical and impulse stochastic control of the exchange rate using interest rates and reserves,” Mathematical Finance, vol. 10, no. 2, pp. 141–156, 2000.

M. Jeanblanc-Picqué, “Impulse control method and exchange rate,” Mathematical Finance, vol. 3, pp. 161–177, 1993.

R. Korn, “Some applications of impulse control in mathematical finance,” Mathematical Methods of Operations Research, vol. 50, no. 3, pp. 493–518, 1999.

B. M. Miller and E. Y. Rubinovich, Impulsive Control in Continuous and Discrete-Continuous Systems, Kluwer Academic, Dordrecht, The Netherlands, 2003.

Z. Wu and F. Zhang, “Stochastic maximum principle for optimal control problems of forward-backward systems involving impulse controls,” IEEE Transactions on Automatic Control, vol. 56, no. 6, pp. 1401–1406, 2011.

Z. Wu and F. Zhang, “Maximum principle for stochastic recursive optimal control problems involving impulse controls,” Abstract and Applied Analysis, vol. 2012, Article ID 709682, 16 pages, 2012.

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, vol. 43 of Applications of Mathematics, Springer, New York, NY, USA, 1999.

I. Karatzas and S. E. Shreve, Methods of Mathematical Finance, Springer, New York, NY, USA, 1998.