Abstract and Applied Analysis

Volume 2014 (2014), Article ID 438258, 11 pages

http://dx.doi.org/10.1155/2014/438258

Research Article

Integration by Parts and Martingale Representation for a Markov Chain

Tak Kuen Siu1,2

¹Cass Business School, City University London, 106 Bunhill Row, London EC1Y 8TZ, UK

²Department of Applied Finance and Actuarial Studies, Faculty of Business and Economics, Macquarie University, Sydney, NSW 2109, Australia

Received 30 October 2013; Accepted 10 May 2014; Published 2 June 2014

Academic Editor: Shuping He

Copyright © 2014 Tak Kuen Siu. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Integration-by-parts formulas for functions of fundamental jump processes relating to a continuous-time, finite-state Markov chain are derived using Bismut's change of measures approach to Malliavin calculus. New expressions for the integrands in stochastic integrals corresponding to representations of martingales for the fundamental jump processes are derived using the integration-by-parts formulas. These results are then applied to hedge contingent claims in a Markov chain financial market, which provides a practical motivation for the developments of the integration-by-parts formulas and the martingale representations.

1. Introduction

Integration by parts is at the heart of Malliavin calculus and its applications. It is deemed to be useful in mathematical finance, stochastic filtering and control as well as the theory of partial differential equations. Particularly, in mathematical finance, an integration-by-parts formula is useful in hedging contingent claims, numerical computations of Greeks, and portfolio optimization; see, for example, Benth et al. [1], León et al. [2], Imkeller [3], and Fournié et al. [4, 5], amongst others. Indeed, integration-by-parts formulas are one of the key results in a number of works on Malliavin calculus for stochastic differential equations driven by Wiener processes and jump processes. Some examples are Bismut [6], Bichteler et al. [7], Bass and Cranston [8], Norris [9], and Elliott and Tsoi [10, 11] to name a few. These authors adopted the approach to Malliavin calculus pioneered by Bismut [6], where an integration-by-parts formula was established by first considering a “small’’ perturbation of the original process and then compensating the effect of the perturbation by Girsanov's change of measure. For an excellent account of Malliavin calculus and its applications, one may refer to, for example, Nualart [12], Privault [13], and di Nunno et al. [14].

Markov chain is an important mathematical tool in probability theory and has vast applications in diverse fields. For example, in finance and actuarial science, there has been an interest in pricing contingent claims under Markov chain markets; see, for example, Norberg [15] and Elliott and Kopp [16] for bond pricing in a Markov chain market, Song et al. [17] for pricing options in a multivariate Markov chain market, Elliott et al. [18] and van der Hoek and Elliott [19, 20] for pricing options in Markov chain markets, and Norberg [21] and Koller [22] for pricing insurance products in Markov chain models. In statistics, particularly in nonlinear time series analysis, Markov chain plays an important role in studying the stochastic stability and ergodicity of stochastic difference equations; see, for example, Tong [23]. Markov chain also plays an important role in stochastic filtering and control. There is a large amount of literature on the use of Markov chain and related stochastic processes in stochastic filtering and control. Some recent literature is Shen et al. [24], He and Liu [25, 26], Zhang et al. [27], He [28], Siu [29], Ellliott and Siu [30], and Wu et al. [31], amongst others. The monograph by Elliott et al. [32] provided discussions on hidden Markov models and their applications in various fields such as signal processing and image processing. The monographs by Yin and Zhang [33, 34] provided discussions on the theories and applications of discrete-time and continuous-time Markov chain, respectively. A recent monograph by Ching et al. [35] presented applications of Markov chain in diverse fields such as manufacturing systems, marketing, and finance.

It appears that in the finance and actuarial science literature much attention has been given to pricing contingent claims in Markov chain markets. It seems that relatively less attention has been paid to hedging contingent claims in Markov chain markets. An integration-by-parts formula is a useful tool for hedging contingent claims. It seems that the literature mainly focuses on developing and applying integration-by-parts formulas in the cases of Wiener processes, Lévy processes, and single jump processes (see, e.g., Elliott and Tsoi [10, 11], Nualart [12], Privault [13], and di Nunno et al. [14]). An integration-by-parts formula in the case of a Markov chain seems lacking. Motivated by the hedging problem in Markov chain markets, it may be of interest to derive an integration-by-parts formula which is useful for hedging contingent claims in Markov chain markets.

In this paper, we derive integration-by-parts formulas for functions of a family of fundamental jump processes relating to a continuous-time, finite-state Markov chain using the Bismut measure change approach. The formulas are derived by considering “small’’ perturbations to the jump intensity parameters of the fundamental jump processes, which are then compensated by Girsanov's measure change. Using the integration-by-parts formulas, new expressions for the integrands in representations of martingales for the fundamental jump processes are derived. Firstly, we consider a function of the terminal values of the fundamental jump processes. Then, the results are extended to a function of the integrals with respect to the whole paths of the fundamental jump processes. The function of the path integrals may be considered a canonical form of a random variable which is measurable with respect to filtration generated by the whole path of the Markov chain. No infinite-dimensional calculus of variations is involved in the derivations. Indeed, only finite-dimensional calculus is adopted. The martingale representation results derived here may be useful for hedging contingent claims in the Markov chain financial market developed by Norberg [21], where the dynamics of share prices were driven by the basic martingales of the fundamental jump processes relating to a continuous-time, finite-state Markov chain.

The rest of the paper is organized as follows. Section 2 describes the Markov chain, the fundamental jump processes, and the basic martingales relating to the chain. Section 3 derives the integration-by-parts formula for a function of the terminal values of the fundamental jump processes. In Section 4, the expression of the integrand in the martingale representation is obtained. The results are then extended to a function of the integrals of the whole paths of the fundamental jump processes in Section 5. An application of the martingale representation result to hedging contingent claims in the Markov chain financial market of Norberg [21] is given in Section 6. Section 7 summarizes the paper and suggests some potential topics for future research.

2. Markov Chain, Fundamental Jump Processes and Basic Martingales

The aim of this section is to present some known results in Markov chain, its fundamental jump processes and basic martingales which are relevant to the later developments.

Consider a complete probability space and a finite time horizon , where . Let be a continuous-time, finite-state Markov chain on . As in Elliott et al. [32], we suppose that the state space of the chain is a finite set of standard unit vectors in , where the component of is the Kronecker delta , for each . The space is called the canonical state space of .

To specify the probability laws of the chain , we define a family of rate matrices, or intensity matrices, under , where, for each , . For each with and each , is the instantaneous transition intensity of the chain from state to state at time . Note that for each and each ,

(1), for ;(2), so .

We suppose here that, for each , is a bounded and deterministic function of time .

Let be the -augmentation of the natural filtration generated by the chain . Note that is right-continuous. Then with the canonical state space of the chain , Elliott et al. [32] obtained the following semimartingale dynamics for :

Here is an -valued, square-integrable, -martingale.

For each with , let , where counts the number of transitions of the chain from state to state up to and including time . That is,

is called a family of fundamental jump processes relating to the chain ; is the scalar product in .

Define, for each with , a process by putting

Then it is obvious from the definition that , , are -martingales and is called a family of basic martingales. Indeed these martingales are orthogonal, purely discontinuous, and square-integrable. Furthermore, .

The following lemma gives the semimartingale decomposition for . This result is standard (see, e.g., Elliott [36] and Elliott et al. [32]).

Lemma 1. For each with and each ,

Proof. The proof of this lemma is standard. For the sake of completeness, we present the proof here:

The last equality is due to the fact that the set of all jump times of the chain has zero “’’-measure.

From Lemma 1 and the definition of ,

is an -martingale. Consequently, under , is the intensity process of .

3. Integration by Parts for Functions of Fundamental Jump Processes

In this section we first present small perturbations to the jump intensities of the fundamental jump processes and then compensate the perturbations by a Girsanov-type measure change. The integration-by-parts formula for a “suitable’’ function of the terminal values of the fundamental jump processes is then derived by differentiation. The techniques used to derive the integration-by-parts formula here are adapted to those used in Elliott and Tsoi [10] for deriving an integration-by-parts formula for a single jump process. It seems that the origin of these techniques may be traced back to the work of Bismut [6].

For each with , let be a nonnegative, -a.s. bounded, -predictable process. Then for an arbitrarily small , we define a small “stochastic’’ perturbation to in the direction by putting

We then take

so that

Note that, for each , and , so , , and .

Define, for each with , the jump process by putting

where is defined in Section 2 as follows:

By definition,

is an -martingale. Consequently, has the intensity process under and it is related to as follows:

To simplify the notation, write , for each with and each . Then

The process is called a perturbed process of the fundamental jump process , so we have a family of perturbed processes corresponding to the family of the fundamental jump processes .

For each with and each , let

Define, for each ,

Consider an -adapted process defined by setting

Then by Elliott [37] (see Theorem 13.5 therein),

where is the stochastic exponential of the process ; .

Then, for each ,

Note that by definition , for each , so the process is strictly positive. Furthermore, is an -martingale.

A new probability measure equivalent to on is now defined by putting

The following lemma will be used to derive the integration-by-parts formula.

Lemma 2. The -law of , with , is equal to the -law of , with .

Proof. By a version of Girsanov’s theorem, the process defined by

is an -martingale. Note that

so has the intensity process under . This is the same as the intensity process of under .

Remark 3. The -martingale defined in the proof of Lemma 2 is related to the -martingale as follows:

To simplify our notation and illustrate the main idea, we consider the situation where the chain has two states. In this case, the family of fundamental jump processes relating to the chain is and its corresponding perturbed processes are .

Let be any measurable, integrable, and differentiable function. Note that from Lemma 2 the -law of is the same as the -law of . Consequently,

Here and are expectations under and , respectively.

Write

where is the transpose of a vector, or a matrix, .

Define the following gradient of with respect to :

Then the following theorem gives the integration-by-parts formula.

Theorem 4. For each , let

Write, for each ,

Then for any measurable, integrable, and differentiable function ,

Proof. By a version of Bayes' rule,

Differentiating both sides with respect to and setting give

It is obvious that and that . Consequently,

Now

Then

Hence the result follows.

The following two integration-by-parts formulas are immediate consequences of Theorem 4.

Corollary 5. For any measurable, integrable, and differentiable function ,

Proof. The result follows by putting , for all , in Theorem 4.

Corollary 6. For any measurable, integrable, and differentiable function ,

Proof. The result follows by putting , for all , in Theorem 4.

Remark 7. The integration-by-parts formula in Corollary 5 (Corollary 6) may be interpreted as an integration-by-parts formula obtained by perturbing the intensity () along the direction ().

Remark 8. In Elliott and Kohlmann [38], an integration-by-parts formula for functions of jump processes was developed. Using the concept of stochastic flows, the integration-by-parts formula was derived for functions of the terminal values of jump processes. An advantage of the approach by Elliott and Kohlmann [38] is that the integration-by-parts formula was derived without using infinite-dimensional calculus. The integration-by-parts formula for functions of the terminal values of jump processes has an important application. Elliott and Kohlmann [38] demonstrated how this integration-by-parts formula may be applied to establish the existence and smoothness of the density of a jump process. This is a key area of application of Malliavin calculus. Using the method in Elliott and Kohlmann [38], the integration-by-parts formula in Theorem 4 may be used to establish the existence and uniqueness of the densities of some stochastic processes depending on the fundamental jump processes relating to the chain. This may represent an interesting topic for future research.

Remark 9. In the Markov chain financial market of Norberg [21], the dynamics of share prices are described by the fundamental jump processes relating to a continuous-time, finite-state Markov chain. The integration-by-parts formula in Theorem 4 may be used to hedge contingent claims whose payoffs depend on the terminal values of the share prices in the continuous-time Markov chain market of Norberg [21]. We will discuss this in some detail in Section 6.

4. Martingale Representation Using Integration by Parts

Martingale representation is one of the fundamental results in stochastic analysis and calculus. It has many significant applications in diverse fields such as mathematical finance, stochastic filtering, and control. A crucial question in a martingale representation is to determine the integrand in the representation. This question is of primary importance in many applications of martingale representations. The Clark-Haussmann-Ocone-Karatzas formula was developed to address this question in the case of a Wiener space (see Clark [39], Haussmann [40], Ocone [41], Ocone and Karatzas [42], and Karatzas et al. [43]). Elliott and Kohlmann [44] pioneered the use of stochastic flows to identify the integrand in a stochastic integral in a martingale representation under a Markov diffusion setting. Elliott and Kohlmann [38] extended the approach in Elliott and Kohlmann [44] to the case of a Markov jump process. Elliott and Tsoi [10, 11] adopted integration-by-parts formulas to derive integrands in martingale representations in a single jump process and a Poisson process, respectively. Aase et al. [45] adopted a white-noise approach to Malliavin calculus to establish a white-noise generalization of the Clark-Haussmann-Ocone-Karatzas formula in the cases of multidimensional Gaussian white noise, multidimensional Poisson white noise, and their combination. Di Nunno et al. [46] adopted a chaos expansion approach to Malliavin calculus to establish a white-noise generalization of the Clark-Haussmann-Ocone-Karatzas formula for Lévy processes.

In this section, we apply the integration-by-parts formula obtained in the last section to derive the integrand in a martingale representation for a function of the terminal values of the fundamental jump processes. Though the techniques to be used here are similar to those adopted in Elliott and Tsoi [10, 11], it seems that the formulas of the integrand derived here appear to be new. Again to simplify our notation, we consider here the two-regime Markov chain presented in Section 3.

Note that the filtration generated by the chain is the same as the filtration generated by the family of fundamental jump processes . Then we state the following martingale representation result which was due to Brémaud [47].

Theorem 10. For any real-valued, square-integrable -martingale ,

for some -valued, -predictable process .

Furthermore, we need the following expression for the predictable quadratic variation of , which was derived in Elliott et al. [32].

Lemma 11. Let be a diagonal matrix with the diagonal elements being given by the components in a vector . For each ,

To simplify our notation, let be a matrix-valued process defined as follows:

Note that is the density process of the measure with respect to the Lebesgue measure on and is absolutely continuous with respect to , where is the Borel -field generated by open subsets of .

Then

The following lemma will be used to derive the expressions for the integrand in the martingale representation.

Lemma 12. For each with , the predictable quadratic variation of , namely , is given by

Proof. Recall that

Then

The last equality follows from the fact that the set of all jump times of the chain has “’’-measure zero.

By the martingale representation presented in Theorem 10,

for some -predictable process .

It can be supposed that by subtraction. Then

The integrand is then determined in the following theorem. Though the techniques used in the proof of the following theorem are similar to those used in Proposition 3.5 of Elliott and Tsoi [11], the expressions for the integrand presented below appear to be new.

Theorem 13. Suppose that for each . Then the integrand , where , is determined by

Proof. We only give the proof for the integrand since the integrand can be derived similarly. Firstly, by the martingale representation for , Lemma 12, and the orthogonality of and ,

Then using the integration-by-parts formula in Corollary 5,

For each , let

Then there exists an -predictable projection of such that, for each ,

so that

Furthermore, for any -predictable process ,

Write for the family of subsets of of the forms and , where and for . Note that the predictable -field on the product space with respect to is generated by .

We now take or , where and are the indicator functions of the events and , respectively. Then the integration-by-parts formula in Corollary 5 holds for this . Consequently, the following equality holds for all ’s which are indicators of sets in :

Since the set of all jump times of the chain has “’’-measure zero,

On the set ,

Consequently, for all ’s which are indicators of sets in ,

Note that

(1) generates the -predictable -field on the product space ;(2)the processes and are -predictable.

Then

Consequently, for almost all ,

Then,

on the set .

5. An Extension to a Function of Path Integrals

The integration-by-parts formulas and the martingale representation developed in the previous sections are now extended to a function of the integrals with respect to the whole paths of the fundamental jump processes relating to the chain . This function may be considered a canonical form of an -measurable random variable.

Consider an -measurable random variable which is of the following canonical form:

where is any measurable, integrable, and differentiable function. Note that depends on the whole paths of the fundamental jump processes relating to the chain ; and are nonnegative, -a.s. bounded, -predictable processes as defined in Section 3.

We now define some notation. Write

Then

The following theorem gives an extension to the integration-by-parts formula presented in Theorem 4 for the function .

Theorem 14. For each , let

Then

Proof. The proof of this theorem resembles that of Theorem 4. We only give some key steps. For each , let

Write

By Lemma 2, the -probability law of is the same as the -law of . Then

Differentiating with respect to and setting give

Then the result follows by noting that

Similarly, the following corollaries are direct consequences of Theorem 14.

Corollary 15. For any measurable, integrable, and differentiable function ,

Corollary 16. For any measurable, integrable, and differentiable function ,

We now extend the martingale representation in Section 3 to the function of the path integrals. By the martingale representation in Theorem 10,

for some -predictable process .

Again by subtraction we assume that . Then

The following theorem gives an expression for the integrand in the martingale representation for .

Theorem 17. Suppose that for each . Then the integrand , where , is determined by

Proof. The proof resembles that of Theorem 13. We only need to note the fact that, for all ’s which are indicators of sets in , .

6. An Application to Hedging Contingent Claims

In this section we will discuss an application of the martingale representation result derived in Section 4 to hedge contingent claims in the Markov chain financial market of Norberg [21]. Here we consider a simplified version of the Markov chain market of Norberg [21], where there are two risky shares, namely, and , and the Markov chain has only two states. We also suppose that the market interest rate is zero. In this case, as in Norberg [21], the (discounted) price processes of the two risky shares and under a risk-neutral probability, say , are governed by

where and , for , are non-zero constants; and are -martingales. Note that the two risky shares are correlated since their price dynamics depend on and .

For each , let and let . Then, as in Norberg [21], under the risk-neutral measure , the (discounted) terminal prices and of the shares are given by

Consequently, the vector of the (discounted) terminal prices of the shares is a function of .

We now consider a contingent claim written on the two correlated risky shares and whose payoff at maturity is a function of , say . Two practical examples of contingent claims having payoffs of this form are an exchange option, which is also called a Margrabe option, and a quanto option.

Note that the payoffs of the Margrable option and the quanto option may not be differentiable functions of . To apply the martingale representation result in Section 4 to derive the hedging quantities for the Margrable option and the quanto option, we need to consider approximations of by some “smooth” or differentiable payoff functions of . In the sequel, we suppose that, with a slight abuse of notation, is such a “smooth” or differentiable payoff function of .

Then, the payoff can be written as

for some “suitable’’ measurable, differentiable and integrable function .

Define, for each , a -matrix by

Then the price processes of the two risky shares and under the risk-neutral measure are governed by the following vector-valued stochastic differential equation:

where as defined in Theorem 4.

Suppose . Then, the inverse of exists and is given by

Consequently,

By the martingale representation in Theorem 10,

Then the claim can be hedged perfectly by constructing a dynamic portfolio which invests units of the risky share and units of the risky share at time , for each . The initial investment of the portfolio is , which is the initial price of the claim . Using Theorem 13, and are determined as

We only illustrate here the use of the martingale representation result in Section 4 to hedge contingent claims whose payoffs depend only on the terminal prices of the risky shares in the Markov chain market. The martingale representation result in Section 5 may be used to hedge contingent claims with more general payoff structures in the Markov chain market.

7. Conclusion

An integration-by-parts formula for a function of the terminal values of the fundamental jump processes relating to a Markov chain was first established using the Bismut approach to Malliavin calculus. The formula was then applied to derive a new expression for the integrand in a stochastic integral in a martingale representation. The results were then extended to functions of the integrals with respect to the whole paths of the fundamental jump processes. These functions may be regarded as random variables of canonical forms. Only finite-dimensional calculus was needed in the derivations. Though some complex notations may be involved, the results presented here may be extended to the case of a general -state Markov chain where a set of fundamental jump processes , , , is used. We applied the martingale representation result derived here to hedge a contingent claim written on two correlated risky shares in the Markov chain financial market of Norberg [21].

There are several future research directions based on the results developed in this paper which may be of theoretical and practical interests. The results may be applied to study the existence and uniqueness of densities of jump processes relating to a Markov chain. It seems that this problem is of fundamental importance in filtering and control theory of hidden Markov chains. Martingale representations play an important role in filtering and control. It may be interesting to explore the applications of the martingale representations developed in this paper in filtering and control for stochastic processes relating to Markov chains. The monograph by Elliott et al. [32] provided some discussions on the filtering and control of hidden Markov chains. The results developed here may be extended to develop Malliavin calculus for stochastic differential equations driven by a continuous-time, finite-state Markov chain and Markov regime-switching stochastic differential equations. It may be of practical interest to further explore the use of the martingale representation results developed here to hedge modern insurance products, such as unit-linked insurance products and longevity bonds in the Markov chain market of Norberg [21]. In Bielecki et al. [48], the valuation of credit derivatives in a Markov chain model was discussed. It may be of practical interest to explore the application of the martingale representation results developed here to hedge credit derivatives in the Markov chain model discussed in Bielecki et al. [48].

Conflict of Interests

The author declares that there is no conflict of interests regarding the publication of this paper.

Acknowledgment

The author would like to thank the editor and the reviewers for helpful comments.

References

F. E. Benth, G. di Nunno, A. Løkka, B. Øksendal, and F. Proske, “Explicit representation of the minimal variance portfolio in markets driven by Lévy processes,” Mathematical Finance, vol. 13, no. 1, pp. 55–72, 2003.

J. A. León, R. Navarro, and D. Nualart, “An anticipating calculus approach to the utility maximization of an insider,” Mathematical Finance, vol. 13, no. 1, pp. 171–185, 2003.

P. Imkeller, “Malliavin's calculus in insider models: additional utility and free lunches,” Mathematical Finance, vol. 13, no. 1, pp. 153–169, 2003.

E. Fournié, J.-M. Lasry, J. Lebuchoux, P.-L. Lions, and N. Touzi, “Applications of Malliavin calculus to Monte Carlo methods in finance,” Finance and Stochastics, vol. 3, no. 4, pp. 391–412, 1999.

E. Fournié, J.-M. Lasry, J. Lebuchoux, and P.-L. Lions, “Applications of Malliavin calculus to Monte-Carlo methods in finance. II,” Finance and Stochastics, vol. 5, no. 2, pp. 201–236, 2001.

J. M. Bismut, “Martingales, the Malliavin calculus and hypoellipticity under general Hörmander's conditions,” Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete, vol. 56, no. 4, pp. 469–505, 1981.

K. Bichteler, J.-B. Gravereaux, and J. Jacod, Malliavin Calculus for Processes with Jumps, vol. 2, Gordon and Breach Science Publishers, New York, NY, USA, 1987.

R. F. Bass and M. Cranston, “The Malliavin calculus for pure jump processes and applications to local time,” Annals of Probability, vol. 14, pp. 490–532, 1986.

J. Norris, “Integration by Parts for Jump Processes,” in Séminaire de Probabilités XXII, vol. 1322 of Lecture Notes in Mathematics, Springer, Berlin, Germany, 1998.

R. J. Elliott and A. H. Tsoi, “Integration by parts for the single jump process,” Statistics & Probability Letters, vol. 12, no. 5, pp. 363–370, 1991.

R. J. Elliott and A. H. Tsoi, “Integration by parts for Poisson processes,” Journal of Multivariate Analysis, vol. 44, no. 2, pp. 179–190, 1993.

D. Nualart, The Malliavin Calculus and Related Topics, Springer, Berlin, Germany, 2nd edition, 2006.

N. Privault, Stochastic Analysis in Discrete and Continuous Settings with Normal Martingales, vol. 1982 of Lecture Notes in Mathematics, Springer, Berlin, Germany, 2009.

G. di Nunno, B. Øksendal, and F. Proske, Malliavin Calculus for Lévy Processes with Applications to Finance, Springer, Berlin, Germany, 2009.

R. Norberg, “A time-continuous markov chain interest model with applications to insurance,” Applied Stochastic Models and Data Analysis, vol. 11, no. 3, pp. 245–256, 1995.

R. J. Elliott and P. E. Kopp, Mathematics of Financial Markets, Springer, New York, NY, USA, 2nd edition, 2005.

N. Song, W. K. Ching, T. K. Siu, E. S. Fung, and M. K. Ng, “Option valuation under a multivariate Markov chain model,” in Proceedings of the IEEE Computer Society Proceedings (CSO '10), vol. 1, pp. 177–181, Huangshan, China.

R. J. Elliott, C. C. Liew, and T. K. Siu, “Characteristic functions and option valuation in a Markov chain market,” Computers & Mathematics with Applications, vol. 62, no. 1, pp. 65–74, 2011.

J. van der Hoek and R. J. Elliott, “American option prices in a Markov chain market model,” Applied Stochastic Models in Business and Industry, vol. 28, no. 1, pp. 35–59, 2012.

J. van der Hoek and R. J. Elliott, “Asset pricing using finite state Markov chain stochastic discount functions,” Stochastic Analysis and Applications, vol. 30, no. 5, pp. 865–894, 2012.

R. Norberg, “The Markov chain market,” ASTIN Bulletin, vol. 33, pp. 265–287, 2003.

M. Koller, Stochastic Models in Life Insurance, Springer, Berlin, Germany, 2012.

H. Tong, Nonlinear Time Series: A Dynamical System Approach, vol. 6, Oxford University Press, Oxford, UK, 1990.

H. Shen, Y. Chu, S. Xu, and Z. Zhang, “Delay-dependent control for jumping delayed systems with two Markov processes,” International Journal of Control, Automation, and Systems, vol. 9, no. 3, pp. 437–441, 2011.

S. He and F. Liu, “Finite-time control of nonlinear jump systems with time-delays via dynamic observer-based state feedback,” IEEE Transactions on Fuzzy Systems, vol. 20, no. 4, pp. 605–614, 2012.

S. He and F. Liu, “Adaptive observer-based fault estimation for stochastic Markovian jumping systems,” Abstract and Applied Analysis, vol. 2012, Article ID 176419, 11 pages, 2012.

X. Zhang, R. J. Elliott, and T. K. Siu, “A stochastic maximum principle for a Markov regime-switching jump-diffusion model and its application to finance,” SIAM Journal on Control and Optimization, vol. 50, no. 2, pp. 964–990, 2012.

S. He, “Resilient - filtering of uncertain Markovian jumping systems within the finite-time interval,” Abstract and Applied Analysis, vol. 2013, Article ID 791296, 7 pages, 2013.

T. K. Siu, “A BSDE approach to optimal investment of an insurer with hidden regime switching,” Stochastic Analysis and Applications, vol. 31, no. 1, pp. 1–18, 2013.

R. J. Elliott and T. K. Siu, “Filtering and change point estimation for hidden Markov-modulated Poisson processes,” Applied Mathematics Letters, vol. 28, pp. 66–71, 2014.

Z.-G. Wu, P. Shi, H. Su, and J. Chu, “Asynchronous - filtering for discrete-time stochastic Markov jump systems with randomly occurred sensor nonlinearities,” Automatica, vol. 50, no. 1, pp. 180–186, 2014.

R. J. Elliott, L. Aggoun, and J. B. Moore, Hidden Markov Models: Estimation and Control, vol. 29, Springer, New York, NY, USA, 1994.

G. G. Yin and Q. Zhang, Discrete-Time Markov Chains: Two-Time-Scale Methods and Applications, vol. 55, Springer, New York, NY, USA, 2005.

G. G. Yin and Q. Zhang, Continuous-Time Markov Chains and Applications: A Two-Time-Scale Approach, vol. 37, Springer, New York, NY, USA, 2nd edition, 2013.

W. K. Ching, X. Huang, M. K. Ng, and T. K. Siu, Markov Chains: Models, Algorithms and Applications, Springer, New York, NY, USA, 2nd edition, 2013.

R. J. Elliott, “A partially observed control problem for Markov chains,” Applied Mathematics and Optimization, vol. 25, no. 2, pp. 151–169, 1992.

R. J. Elliott, Stochastic Calculus and Applications, vol. 18, Springer, Berlin, Germany, 1982.

R. J. Elliott and M. Kohlmann, “Integration by parts and densities for jump processes,” Stochastics and Stochastics Reports, vol. 27, no. 2, pp. 83–97, 1989.

J. M. C. Clark, “The representation of functionals of Brownian motion by stochastic integrals,” Annals of Mathematical Statistics, vol. 41, pp. 1282–1295, 1970.

U. G. Haussmann, “On the integral representation of functionals of Itô processes,” Stochastics, vol. 3, no. 1, pp. 17–28, 1979.

D. Ocone, “Malliavin's calculus and stochastic integral representations of functionals of diffusion processes,” Stochastics, vol. 12, no. 3-4, pp. 161–185, 1984.

D. L. Ocone and I. Karatzas, “A generalized Clark representation formula, with application to optimal portfolios,” Stochastics and Stochastics Reports, vol. 34, no. 3-4, pp. 187–220, 1991.

I. Karatzas, D. L. Ocone, and J. Li, “An extension of Clark's formula,” Stochastics and Stochastics Reports, vol. 37, no. 3, pp. 127–131, 1991.

R. J. Elliott and M. Kohlmann, “A short proof of a martingale representation result,” Statistics & Probability Letters, vol. 6, no. 5, pp. 327–329, 1988.

K. Aase, B. Øksendal, N. Privault, and J. Ubøe, “White noise generalizations of the Clark-Haussmann-Ocone theorem with application to mathematical finance,” Finance and Stochastics, vol. 4, no. 4, pp. 465–496, 2000.

G. di Nunno, B. Øksendal, and F. Proske, “White noise analysis for Lévy processes,” Journal of Functional Analysis, vol. 206, no. 1, pp. 109–148, 2004.

P. Brémaud, Point Processes and Queues: Martingale Dynamics, Springer, Berlin, Germany, 1981.

T. R. Bielecki, S. Crepey, and A. Herbertsson, “Markov chain models of portfolio credit risk,” in Oxford Handbook of Credit Derivatives, A. Rennie and A. Lipton, Eds., pp. 327–382, Oxford University Press, Oxford, UK, 2011.