New Method of Sensitivity Computation Based on Markov Models with Its Application for Risk Management

Sensitivity analysis is at the core of risk management for financial engineering; to calculate the sensitivity with respect to parameters in models with probability expectation, the most traditional approach applies the finite difference method, whereafter integration by parts formula was developed based on the Brownian environment and applied in sensitivity analysis for better computational efficiency than that of finite difference. Establishing a similar version of integration by parts formula for the Markovian environment is the main focus and contribution of this paper. It is also shown by numerical simulation that our proposed methodology and approach outperform the traditional finite difference method for sensitivity computation. For empirical studies of sensitivity analysis on an NPV (net present value) model, we show the approaches of modeling, especially for parameter estimation of Markov chains given data of company loan states. Applying our newly established integration by parts formula, numerical simulation estimates the variations caused by the capital return rate and multiplier of overdue loan. Fur-thermore, managemental implications of these results are discussed for the effectiveness of modeling and the investment risk control.


Introduction
Sensitivity analysis studying the variation of value function with respect to the value changes of certain parameter plays a vital role in risk management in derivative market, especially for portfolio pricing and hedging (cf. Haimes [1]). As is often achieved by estimating the Greeks, price sensitivities related to variations of model parameters are calculated and investigated. In particular, Greeks are the sensitivities with respect to parameters in the pricing model of project investment, where sensitivity analysis guides parameter estimation of the modeling.
Starting from Fournié et al. [2], Malliavin calculus is applied for computation of Greeks such like Delta(Δ) and Rho(ρ), respectively, representing the sensitivities of option value with respect to spot price and riskfree rate (also refer to Davis and Johansson [3]). rough these approaches, fast algorithms for Greeks computations are designed. However, this technique is established only for the Brownian models, or at most based on the randomness of Brownian motion for those models with mixed sources of randomness. Moreover, according to the best of our knowledge, the essence of the technique relies on an integration by parts formula of Wiener-Malliavin calculus. erefore, extension of the existing technique for more extensive applications is subjected to new versions of integration by parts formula of other stochastic processes.
us, emerging interests focus on the establishment of integration by parts formula of Markov chains, and multiple versions of integration by parts formula are achieved under di erent assumptions and conditions by Siu [4], Denis and Nguyen [5], and Liu and Privault [6], respectively.
On the other hand, Markovian models are widely applied to structure engineering and nancial problems with multistage switching, among which loan portfolio and investment risk are efficiently modeled with Markov chains whose states indicate the loan ratings. One significant application is for the study of credit migration matric to predict the credit risk of a financial institutions portfolio (cf. Jafry and Schuermann [7]; Jones [8]; and Schuermann [9]). Adequacy evaluation and analysis of Markovian model for loan rating are proceeded in line with improvements of statistical treatments of data (we refer to Kieferand Larson [10] for more discussion and Anderson and Goodman [11] for more backgrounds). Along this direction, alternatives of the standard Markovian models are also developed for loan quality analyses, for instance, hidden Markov chains were considered by Quirini and Vannucci [12] and double Markov chain was applied in Kaniovski and Pflug [13]. For the loan quality analyses, the Markovian model is applicable to capture the repayment behavior, as in yagarajan and Saiful [14]; hence, it predicts the return of some receivables under considerations of loan default.
In particular and for evaluation of an investment project, income flow can be reflected and investigated by some quantitative indicators of financial analysis. NPV (net present value) is one efficient indicator for this end. As proposed in Briggs and Sculpher [15] and later developed in van Der Laan [16], Markovian models are widely applied to govern the underlying state changes in the expression of NPV in Forsell et al. [17]; Timofeeva and Timofeev [18]; Timofeeva [19]; and so on. erefore, it is necessary to investigate the sensitivity of the Markovian models of NPV estimation.
In this paper, we develop fast computational approach for sensitivity analysis. In particular, closed-form expressions of sensitivity for two major classes of Markovian models are concluded by applying the integration by parts techniques (refer to Siu [4]). To the best of our knowledge and generally speaking, finite difference was the unique methodology for sensitivity computations of the pure Markovian model. Paralleling to the application of integration by parts formula of Winner-Malliavin environment, our technique pioneers a new method to compute the sensitivity of the pure Markovian model without using the finite difference. Other possible alternatives are very likely to be developed through the recent theoretical results about integration by parts formula in Denis and Nguyen [5] and Liu and Privault [6]. e remainder is arranged as follows. Section 2 formulates the sensitivity analysis of the NPV model based on Markov chains. Section 3 states the main results of sensitivity computation, for which the mathematical proofs are given in Appendix A. Applying the formula in Section 3, sensitivity analysis of the above-mentioned NPV model is proceeded numerically with simulated parameters in Section 4, based on which, we present the practical guidelines of applying this model for risk management and conclude the whole article. By Appendix, we also show the technique of determining transition rate matrix of Markov chain with real data from market surveys. Finally, we conclude this paper in Section 5 and propose some possible work associated with this paper.

A Practical Problem Motivating for Technical Improvements of Sensitivity Analysis
As introduced above, cash flow management intensively demands for the calculation and prediction of NPV value, which is formulated as follows: where r is the required rate of capital return and I i denotes the income during the time period Note that I 0 < 0 is the initial investment, and t n � T is the time horizon of this investment.
As we noticed, Timofeeva and Timofeev [18] and Timofeeva [19] applied a time-discrete Markov chain with 5 states to capture the dynamics of income flow. However, it is quite possible to observe state switching between the discrete time points; therefore, in this paper, a time-continuous Markov chain is used to simulate the changes of underlying situations, which are determined by the progress of loan repayments. In particular, a Markov chain β t t∈[0,T] is defined for this sake, and similar to the setting of Timofeeva and Timofeev [18] and Timofeeva [19], 5 states of the state space M ≔ 1, 2, 3, 4, 5 { } are given as follows: (1) No overdue loans.
(2) Small amount of short-time overdue loans. Remark 1. In the above statement, we need to stress that the so called "short time" and "long time" are relevant to how many days it has been overdue till now, not the term structures of loan. To classify according to the time span, we refer to the average length of overdue period (for those finally cleared loans) in the same industry and economic zone. More details are given in the case shown in Section 4. Similarly, for the classification of "small amount" and "large amount," we also refer to the average amount of loans in the same industry and economic zone.
Intuitionally and in some situations, we may feel that the possibility of transition from state "2" to state "1" is greater than that from state "5" to state "1." Observations like this will be reflected on the transition probability rate matrix Q of the Markov chain, obtained by surveys in the same industry. To transform the Markov states to some numerical values, we define a real-valued function f(·) on M that

Remark 2.
is function f(·) maps each state of Markov chains to its corresponding real-valued number, which becomes one factor in the expression of income I i at time t i for i ∈ 1, . . . , n { }. Since I i will be affected by the absolute amount of overdue loan at time t i and independent of the time span of being overdue, we let f(2) � f(3) and f(4) � f (5). Besides, we need a multiplier σ placed before this function f(·), and hence the income loss caused by small amount of overdue loans is σ and that by large amount becomes 3σ. Denote the average amount of overdue loans as M, and the average of small amount overdue loans is about M/2 and that of large amount is about 3M/2; therefore, we approximately let the income loss caused by large amount of overdue triple that of small amount.
To simulate the income flow I i i∈N , we let a deterministic sequence b i i∈N denote lower bound of expected income, let a deterministic sequence a i i∈N such that a i > b i for each i denote the expected income without considerations of loan delinquencies, and let a constant σ > 0 denote the multiplier of overdue loan; we call it influence coefficient, and hence the income flow can be formulated by and for any n ∈ N + , the expected NPV is given by Remark 3. In expression (2) of income flow, the expected income without considerations of loan delinquencies, denoted as a i , is unforeseen in this model. It should be precisely predicted based on the financial and accounting information of projects and will partially affect the precision of our sensitivity analysis. But it is not the focus of this paper and we assume that the predictions of a i by decision makers and managers of enterprise are believable, and hence the main uncertainty comes from the loan quality and repayment behavior. Another sequence of variables b i is relatively easier to be estimated according to historical data and empirical evidences. Feeding the model with this lower bound information by b i will reduce some unexpected highlevel randomness and statistical errors. Based on these considerations, we formula the income flow as (2). e above expectation of NPV provides the project managers with numbers of meaningful insights for strategic management, cost accounting, and risk management. As mentioned above in the introduction, sensitivity analysis is a crucial means to govern the potential risks caused by various factors. Specifically, this model (3) is probably subject to the capital return rate and influence coefficient σ of state changes; therefore, investigation on the sensitivity analysis with respect to r and σ plays a vital role on controlling the risk of interest rate and situation changes of overdue loans, respectively. Besides, this sensitivity analysis will provide quantitative criteria of model stationary adequacy and improvements of accuracy by practically optimizing the parameter determinations. In particular, we define the sensitivity of E npv (r, σ) with respect to σ ∈ R + as and similarly define the sensitivity with respect to r ∈ R + as at the point that r � r 0 , σ � σ 0 .

General Expression of Sensitivities Based on Two Classes of Markovian Models
In order to compute the sensitivity of the expected NPV mentioned above, we aim to develop a setting of closed-form formulas of sensitivity for two general classes of Markovian models. Some main results of it will be applied to establish a closed-form expression of (5).

Mathematical Expressions.
Starting with a probability space Hereafter, we consider two classes of processes (X t ) t∈R + driven by Markov chain α t t∈[0,T] and in form of the expressions below, for which we will analyze the sensitivity regarding any specific parameter θ ∈ R of their models. We have class A that where F(t, x, θ) on [0, T] × M × R is twice continuously differentiable with respect to x and θ, satisfying the condition that the set x|F x ′ (T, x, θ) � 0 is countable and for

Remark 4. Classes
A and B cover almost all commonly used functionals of Markovian processes except those mixed with extra sources of randomness such as Brownian motions.
Next, consider the value function V(α 0 , θ) defined below: where ϕ is a differentiable function with bounded derivative and . en, we have the sensitivity of V(α 0 , θ) with respect to the parameter θ, stated in Propositions 1 and 2, whose proofs are attached in Section 3.2.

Proposition 1. For any real-valued differentiable function ϕ(·) with bounded derivative and (X (θ)
t ) t∈[0,T],θ∈R in the class A defined by (7), we have where x, θ)/zx and zF(t, x, θ)/zθ, respectively, and for any i ∈ M, and for any i, j ∈ M, t ∈ [0, T], and For the convenience of notations, we denote For any (9) with respect to the parameter θ ∈ R is given by Proposition 2.
By the same approach as in Kawai and Takeuchi [20] and Liu and Privault [6] (Section 6, Pages 933-935), we extent the results in Propositions 1 and 2 for nondifferentiable function ϕ ∈ Λ(R; R) defined as follows: where us, the following proposition summarizes our all results.

Proofs of the above Propositions.
In this section, we prove Propositions 1 and 2.
,θ∈R in the form of (7), sensitivity of V(α 0 , θ) in (9) with respect to the parameter θ ∈ R is given by Proposition 1, proved as follows.

Proof. Given a differentiable function H:
and for any r.
We say a r.v. β ≔ H(J 1,2 (T), J 2,1 (T), . . . , J m,m− 1 (T)) is differentiable by D ] when H is differentiable, and it is denoted as: β ∈ Do m(D ] ), and thus we define Because of the equation that where erefore, and the function ϕ (F(T, x, θ) where λ i,j (t) ≔ 1 α t �i { } /(i − j) for any t ∈ [0, T], i ≠ j ∈ M. Hence, with the gradient operator D (·) , we have the integration by parts formula of Markov chain. By eorem 1 in and the chain rule for integrable and differentiable function H, K: Note that ϕ(X (θ) is integrable, and since z/zθX (θ) T is bounded, the order of differentiation w.r.t θ and expectation is changeable. By Definitions (19) and (20) □ Remark 5. Besides λ defined in (26), alternatives for ] in the operator D ] are also feasible; a process π defined by will give another version of integration by parts formula: given a differentiable and integrable function H on R m(m− 1) , we obtain with

Lemma 1. For any real-valued function f on
where J i,j (t) i,j∈M,t∈ [0,T] are defined by (12) and (13).

Numerical Simulation.
In this section, we proceed with a numerical simulation to compare our approaches stated by Proposition 2 with those by finite difference for sensitivity analysis. Note that numerical simulation for Proposition 1 can be similarly done since class A is of similar expressions. Consider the integration of a Markov chain that where the Markov chain (α t ) t∈R + on the state space M � 1, 2 { } has a Q � (q i,j ) 2×2 matrix that q 1,1 � − λ < 0 and q 2,2 � − μ < 0. A value function V is defined by where ϕ(x) ≔ 1 x>K { } . For this simulation, we arbitrarily let λ � 0.5, μ � 0.4, T � 10.0, K � 46.0, f(1) � 0.5, and f(2) � 0.4. en, the sensitivity of V(α 0 , θ) with respect to θ is expressed as the derivative of V(α 0 , θ) w.r.t. θ. In particular, we consider the derivative at α 0 � 1, θ � 10, that is, Applying both methods to calculate the sensitivity, the computational results of Θ converge to 0.45 approximately, as shown in Figure 1 of four numerical trials (same parameters) to observe convergence of numerical results by enlarging the sample size, and these observations are sufficient to conclude that the results generated by our method converge faster than those by the finite difference method.
On the other hand, we repeat the comparison with another transition rate matrix Q that To proceed the test in a much stabler environment of randomness, we find that the computational results of both method converge to 0.52 and the convergence speed of our approach is even faster. e sensitivity value is larger because the initial state is "1" and f(1) > f (2), and hence a stable environment makes it stay in "1" for longer time; by checking the expression of X (θ) T , we see that the factor multiplying θ having a longer time to be larger leads to a larger sensitivity of θ.
Although it seems not conclusive by comparison of both methods with merely two settings of parameters, we also have statistical reason accounting for this phenomenon. As is well known, the finite difference method provides a biased estimator of Θ in (48): for some small ε > 0, such as 0.001. By contrast, our estimator given by (15) is an unbiased one, and hence it is more likely to obtain a precise result by faster convergence. Paralleling to this, Fournié et al. [2]; Siu [4]; Denis and Nguyen [5]; and Liu and Privault [6] applied a similar technique of integration by parts but based on the randomness of Brownian motion, and all of them show better computational efficiency and precision than finite difference. ese evidences show great advantages of the integration by parts technique for the computation of sensitivities.

Remark 6.
Hence, it is sufficient to claim that our newly developed approach outperforms all existed methods for the case with discontinuous payoff function ϕ(·), since in the past it was only computable by applying the finite difference method. In case that ϕ(·) is sufficiently smooth, (48) is computed by which could be easier to calculate than our approach does. However, in practice, most models will not satisfy the conditions of smoothness for mathematical conveniences. Moreover, for the computation based on our model, (2) is also discontinuous with respect to σ.

Application of Results in Section 3 for the Problem in Section 2
In this section, we show one simulated example of sensitivity analysis mentioned in Section 2 which is solved by applying the formula in Section 3. Although this example regarding its specific industry will not universally conclude to this modeling for all applications to multiple areas, it also enlightens us on the practices of applying this model for sensitivity analysis and risk management on real market. Constructed in Section 2, the expectation of NVP (defined in (1)) is denoted as E npv (r, σ) and has the expression (3), where r is the required rate of capital return and σ is the influence coefficient of the overdue loan state changes. By expression (4) of E npv (r, σ), S r (r 0 , σ 0 ) defined by (6) is easily obtained as follows: However, S σ (r 0 , σ 0 ) defined in (5) cannot be tackled similarly since the order exchange of differentiation and expectation in is invalid for any t ∈ (0, T] and constant real numbers b and a, and max(b, a − σf(β t i )) is even nondifferentiable regarding σ. In this situation, Proposition 3 meets the exact need, as applied in the following approaches. Let , m � 5 in the formula (10), and consider f(β t ) as another Markov chain with the same transition probability rate as those of β t ; formula (10) yields Journal of Mathematics 9 where J i,j (t) i,j∈M,t∈ [0,T] are defined by (12) and (13) and T i (t) count the total time of staying at state i up to time t. It follows that Consider an investment project with receivables in form of loans; it has n � 10 terms during the time horizon [0, 1] (let T � 1.0), and the expected incomes (let the unit "1" denote 1 million dollars) a i and expected lower bounds b i of incomes for each term i ∈ 1, . . . , 10 { } are given in Table 1. Since it is for one single project considered and these data should be very specific, we suggest not to resort to external information from other enterprises in the market, and project managers should be capable to provide convinced values of a i and b i according to the project schedule as well as their empirical predictions.
Besides, investigation by market survey determines the parameters in the transition rate matrix Q 2 (refer to the Appendix for more details):    Journal of Mathematics According to (52) and (55), we compute the sensitivity values of S r (r 0 , σ 0 ) in (6) and S σ (r 0 , σ 0 ) in (5) for different values of (r 0 , σ 0 ), which are plotted as two surfaces in Figure 2.
In practice, these computational results as well as the graph above are meaningful and instructional. For instance, S r (0.05, 0.1) � − 19.5 denotes the sensitivity of E npv regarding r when r � 0.05 and σ � 0.1, and it illustrates that   small change of this rate, say Δr � 0.01, may bring about the value change 0.2 of E npv when we consider the case (r, σ) which is about (0.05, 0.1). is information is crucial for us to control the computational error balanced with the cost of raising the accuracy of parameter r. For this sake, comparison between S r (r 0 , σ 0 ) and S σ (r 0 , σ 0 ) as shown in Figure 2 is also meaningful. In the most value ranges of (r, σ), the absolute value of sensitivity regarding r is larger than that regarding σ unless r is very small. Two implications are obtained from the statistical observations: (1) For the improvement of computational efficiency and accuracy, it is suggested to consume more efforts on determining the capital return rate r while the value assignment of σ is not overwhelmingly influential in this case, and hence less efforts are required for this. (2) When the capital return rate r becomes very small (less than 0.02), the NPV value will be very sensitive to the influences of state changes of overdue loans, that is, deterioration of loan state will cause a sharp cut of NPV in this situation. e managerial implication of this result is that extra protective procedures might be very essential to react against sudden changes of loan states for sake of capital risk control when the return rate r is below certain level.

Conclusion and Summation
is paper develops fast computational approach for sensitivity analysis and establishes the closed-form expressions of sensitivity for two major classes of Markovian models by applying the integration by parts technique, which is applied for the sensitivity analysis of an industrial case regrading its investment projects. Main contributions of this paper include the following. For one thing, it introduces the Markovian model for the NPV estimation, especially for its sensitivity analysis. Five states of time-continues Markov chains (while most existing models for this were time-discrete) well describe the loan states. is advance will be developmental for research on this topic. For another, the mathematical work is with deep skills and high potential to be widely applied for computations of Markovian models, since this integration by parts technique of Markov chain was newly developed and Section 3 concludes two very general cases.
However, there is some room of improvement and much can be done for further studies. As for the mathematical work of developing the Markovian integration by parts technique, we wish to extend the results for a more general class of Markovian process or processes with Markov chains.
Moreover, we suspect whether it is possible to apply this method for the Brownian environment. As for the application of the sensitivity analysis based on the Markovian model, we believe that it can also be widely applied in the research of option pricing and hedging; hopefully, it will be achieved by researchers in the future.
In this appendix, we show the approaches of value assignment of transition rate matrix for one instance. In this practical case of example, we aim to proceed the sensitivity analysis ( (5) and (6)) based on our NPV models (3) and (4) for a listed manufacturing company DSJM (002384) in Jiangsu Province, China. e whole project period of each is denoted as T � 1, and let n � 10; hence, we observed and recorded the loan states for each 10 percent portion of the project period according to our surveys; the states are marked as in Section 2 where 1 denotes that there are no overdue loans, 2 denotes small amount of short-time overdue loans, 3 denotes small amount of long-time overdue loans, 4 denotes large amount of short-time overdue loans, and 5 denotes large amount of long-time overdue loans. e following statistical knowledge is applied to transform the information in Table 2 to the transition rate matrix. us, we let q i,i � − 1/τ(i) for each i ∈ M. e remaining work is to determine q i,j for i ≠ j, so we let s i,j denote totally how many times the event β τ(i) � j happens during those projects shown in Table 2. It follows that the matrix S: � s i,j is obtained: S � Based on that fact that 5 j�1 q i,j � 0 for any i ∈ M and formula (A.3), we assign q i,j to be − s i,j q i,i / 5 j�1,j ≠ i q i,j . Hence, we obtain (56).

Data Availability
e data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
e authors declare that they have no conflicts of interest.