A Comparative Study of VaR Estimation for Structured Products

The previous literature commonly concluded that GARCH models provide better volatility forecasts in financial markets. This paper adopts the backtesting criteria, the multivariate extension of the Diebold and Mariano (1995) test, RMSE (root mean squared errors), and MAE (mean absolute errors) to compare the performances of selected conditional heteroscedastic models for structured products in the low oil price and high oil price periods. The results illustrate that, in general, the performance of GARCH type seems to be better in the whole periods whereas it is not in the low period and the high oil price period.


Introduction
Structured products include reverse convertible bonds, discount certificates, and principal-protected notes. They can prevent investors from market risk and credit risk. Telpner [1] illustrates that depending on volatility of asset returns and the spread of long rate and short rate, investors can choose a suitable investment tool. Principal-protected notes are widespread securities in structured products. These notes can protect investors' final payoffs in recession and become an important instrument in portfolio selections.
Principal-protected note prices can be linked to stock index, interest rate, exchange rate, and commodities. Salomon Brothers Inc. first issued stock index-linked and principal-protected notes in 1986, namely SPIN, whose value was linked to S&P500. A year later, Chase Manhattan Bank launched a market index linked to certificates of deposit, called MICD, which was the first one of its kind by a commercial bank. In 1991, Goldman Sachs & Co. developed SIGN also linked to S&P500 in Australia.
In the 1990s, the fashion of principal-protected notes is increasing in portfolio selections. Hong Kong Investment Funds Association reported that their increase percentage had accounted from 2.94% to 84% in the 2001∼2002 period. In Taiwan, the notes started in 2003 due to legal restrictions but began to pick up momentum by Winbond Electronics.
As for academic research of the principal-protected notes, the seminal work by Brennan and Schwartz [2] ushered in the concept of convex products and protected put products. The research of convex products can be traced to Gerber and Pafumi [3] and Imai and Boyle [4]. They investigate the dynamic hedging strategy in continuous time.
The literature in the framework of a protected put strategy follows the line of Chance and Broughton [5], Chen and Kensinger [6], and Chen et al. [7]. Chance and Broughton [5] and Chen and Kensinger [6] set the floor for stock growth rate and calculated the fair price of the MICD in the absence of the maximal payoff. Chen et al. [7] examine the value of investment funds linked to the Asian and Latin American stock indices.
Alternatively, the majority of the empirical studies focus on the relationship between the theoretical and actual prices of financial products. For example, Chen and Sears [8] analyze the difference of the two prices on SPIN. Burth et al. [9] computed the difference of the two Swiss structured products based on the root mean squared error. Additionally, Wilkens et al. [10] estimate the impact of actual price differentials and maturity date on the average price differential by means of regression analysis on reverse convertible bonds and discount certificates.
Above all, we find that no literature of structured products focuses on the management of market risk. In general, the VaR measure is commonly used to quantify market risk. Some available methods are the following: Riskmetrics, the GARCH approach, quantile estimation, and extreme value theory (for instance, McNeil and Frey [11] 2 Economics Research International and Zivot and Wang [12]). In spite of any approach, forecast volatility is important to VaR calculation. Previous studies, such as the works of Akgiray [13], Pagan and Schwert [14], Baillie and Bollerslev [15], West and Cho [16], Brooks [17], and Taylor [18], generally evaluated asset returns' volatility by different time series models and measured the accuracy by mean absolute squared errors (MAEs) or mean squared errors (MSEs). In all these studies, the errors are represented as the absolute gap between actual volatility (measured by a squared daily return) and model volatility. A general conclusion from these studies is that GARCH models tend to perform better in measuring volatility than the other time series models. In contrast to MAE and MSE tests, the Diebold and Mariano [19] test allows for forecast error that can be non-Gaussian, nonzero mean, serially correlated, and contemporaneously correlated for nested models. It can satisfy a real life.
In addition, Jorion [20] shows that VaR measure only can be efficiently used in a normal market. But, in the recent years, the world economy is not stable. One of the reasons is owing to oil price fluctuation, which causes stock price fluctuation. Many previous studies show that the impact of oil price changes on the world economy is large such as the studies of Adelman [21] and Driesprong et al. [22]. Driesprong et al. [22] find that stock returns tend to be lower after oil price increases and higher if the oil price falls in the previous month. Since West  The purpose of this paper is to reexamine the forecast accuracy of GARCH by means of the multivariate extension of the Diebold and Mariano [19] test, which is proposed by Mariano and Preve [23] for nonnested models, and backtesting method using index-linked and principal-protected notes as samples, which are traded on the American Stock Exchange from May 13, 2004, to April 30, 2008. The time break of August 30, 2005, is chosen to make a comparative study in low oil price and high oil price periods. The last section contains a conclusion.
Index-linked and principal-protected notes are one of structured products. They have some properties. We can take an example to describe them. Assume that the initial (notional) par value of the notes is N 0 per unit. Issuers promise at expiration (T) to deliver capital gains (N T − N 0 ) and (1 + λ) portions of the initial par value if the par value at maturity N T is no fewer than N 0 , where λ is a constant guaranteed rate. Otherwise, issuers only pay (1 + λ) portions of the initial par value if the par value at maturity is less than the initial par value.
Hence, the final payoff of the notes, denoted by P T , can be written as Take an examination of (1). One knows that the payoff of the structured products can be decomposed into an insured amount, (1 + λ )N 0 , and a call option, Max[N T − N 0 , 0]. Obviously, investors can obtain a minimum amount of (1 + λ )N 0 in Figure 1. Furthermore, the par value of the notes monitored at time t i , N ti , depends on a stock index return at observed time, t i , for i = 1, 2, . . . , n. The time interval of [t i−1 , t i ] is set to be equal. Note that t n = T and t 0 = 0. Within this framework, the par value at time t i is affected by the stock returns at the time. Again, it is expressed as Economics Research International 3 where S ti is the stock index at time t i and γ is a constant participation rate. Finally, the payoffs of the notes at time t i are also expected to be Hence these notes are called index-linked and principalprotected notes. Practically, in Taiwan, the gross guaranteed rate (1+λ) is generally more than 80%, and participation rate is less than 1 but more than 90%.

Model Checking
Consider the RUD, DJQ, and SPO notes, which are index-       Note that RMSE and MAE demonstrate root mean squared errors and mean absolute errors, respectively. Table 1 reports that the skewness of RUD and SPO notes is negative during the three periods whereas that of DJQ notes is positive. Additionally, the kurtosis of RUD, SPO, and DJQ notes is fewer than 3, namely platykurtic kurtosis, during the low oil price period. Conversely, the kurtosis is leptokurtic for these three kinds of notes in the high oil price and whole time. Alternatively, Ljung-Box statistic with 10 lags, Q(10) of log returns, is not significant. However, evidence of time dependence is found using the Q(10) of squared log returns, which is robust to heteroscedasticity in the last column of Table 1. Also, Figures 2(a), 2(b), and 2(c) show that the sample autocorrelation functions (ACFs) of the daily log returns of RUD, DJQ, and SPO notes are with lower serial correlations at significant level of 10% while the sample ACFs of the squared daily log returns of these notes demonstrate that these returns are significantly correlated and have some ARCH effects as illustrated in Figures 3(a), 3(b), and 3(c). This also means that large shocks tend to be followed by another large shock. The feature implies that the volatility of the asset returns exists heterogeneity. Hence, some ordinary conditional heteroscedastic models-AR(1)-ARCH(p), AR(1)-GARCH(p, q), and AR(1)-EGARCH(p, q) models, are employed to perform model checking in this paper. These models can be written as follows: (1) AR(1)-ARCH(p): (2) AR(1)-GARCH(p, q): (3) AR(1)-EGARCH(p, q): In the above models, r t denotes the daily log returns at time t, ε t stands for the shock at time t, and μ 0 , μ 1 , α i , β i , and γ represent constant coefficients.
Then, the order of these models is determined by means of AIC and SBIC. AIC and SBIC denote Akaike information criterion and Schwartz Bayesian information criterion, respectively. As stated in Table 2 The estimation of the selected models is shown in Table 3 during the three periods-the whole period, the low oil price period, and the high oil price period. In next section, the results of the estimation of these selected models are used to measure VaRs of index-linked and principal-protected notes.

Accuracy Evaluation
This section will conduct accuracy evaluation of selected time series models for VaR forecast by backtesting method, the multivariate extension of Diebold and Mariano test, RMSE, and MAE. Before backtesting method, it is necessary to calculate VaRs for these three kinds of securities.

VaR Measurement and Implication.
Daily returns of RUD, SPO, and DJQ are used in these samples. They are computed using the following formula: where P t is the market value of the notes at time t. VaR can be defined as the maximal loss over a target horizon (l) with a given probability (α). Assuming normal returns of the asset, it can be written as where E[r t (l)] denotes the expected return over an l-period horizon, Z α stands for a critical value with a given probability α, and σ t (l) is the forecast volatility of returns. In this paper, the VaRs are for a one-day-ahead horizon and a 99% confidence level for losses such that l = 1, α = 1%, and Z α = −2.33. Applying Table 3, one can compute VAR(1) summarized in Table 4. Table 4 indicates that the VaRs based on GARCH-type models are the smallest for RUD notes in the whole period and low oil price period; the VaRs in AR(1)-ARCH(5) model are the smallest in high oil price period. This implies that in the GARCH-type models, it is necessary for financial investors to keep the greatest need of a minimum solvency safety margin because the worst potential losses of the models are the largest than the other models in the whole period and low oil price period. Meanwhile, the result is low profitability of the return on equity (ROE) and downward pressure on the bank's stock price during the whole and low oil price periods. Similarly, the phenomena occur in the AR(1)-ARCH(5) model in the high oil price period.
As for SPO notes, the minimum VaRs happen in the GARCH-type models during the low and high oil price periods; the VaR of the AR(1)-ARCH(5) model is the lowest among those of the other models in the whole period. Hence, it is necessary for a bank to maintain the maximum capital amount in order to prevent from default risk in terms 8      Note that Y1 and Y3 stand for the differences of the sum of squared errors (SSEs) of volatility in GARCH from AR(1)-ARCH(5) and AR(1)-EGARCH(1,1), respectively. Y2 denotes the differences of SSE of volatility in AR(1)-GARCH(1,1) from AR(1)-GARCH(1,4) as the benchmark is AR(1)-GARCH(1,1). Conversely, when the benchmark is AR(1)-GARCH(1,4),Y2 represents the differences of SSE of volatility in AR(1)-GARCH(1,4) from AR(1)-GARCH(1,1). Z1 and Z3 stand for the differences of SSE of volatility in AR(1)-GARCH from AR(1)-ARCH(5) and AR(1)-EGARCH(1,1), respectively. Z2 denotes the differences of SSE of volatility in AR(1)-GARCH(1,1) from GARCH(2,5) as the benchmark is GARCH (1,1). Conversely, when the benchmark is GARCH(2,5), Y2 represents the differences of SSE of volatility in AR(1)-GARCH (2,5) from AR(1)-GARCH(1,1).W1, W3, and W4 stand for the differences of the sum of SSE of volatility in AR(1)-GARCH from AR(1)-ARCH(5), AR(1)-EGARCH(2,1), and AR(1)-EGARCH(2,2), respectively. W2 denotes the differences of SSE of volatility in AR(1)-GARCH(2,1) from AR(1)-GARCH(2,5) as the benchmark is AR(1)-GARCH(2,1). Conversely, when the benchmark is AR(1)-GARCH(2,5), Y2 represents the differences of SSE of volatility in AR(1)-GARCH(2,5) from AR(1)-GARCH(2,1).The symbol * denotes significance at 5 percent level.  Note that W, L, and H stand for the whole period, the low oil price period, and the high oil price period, respectively. NB and B denote "not better" and "better", respectively. RMSE and MAE demonstrate root mean squared errors and mean absolute errors, respectively.  of the GARCH-type models during the low oil price and high oil price periods. However, the need of a minimum solvency safety margin under the AR(1)-ARCH(5) model is the greatest during the whole period. Again, for DJQ notes, the VaRs measured by the EGARCH-type models are the smallest in the whole and high oil price periods; the capital requirement of the AR(1)-GARCH(2,1) model is the largest in the low oil price period.

Economics Research International
To sum up, there exists a consistent phenomenon that the VaRs of GARCH-type models are the smallest among those of the selected models during the low oil price period. When financial risk managers employ GARCH-type models to measure VaRs during the low oil price period, the firm's default risk can fall because of sufficient capital requirement. In general, the worst potential losses of ARCH-type models are fewer than those of GARCH-type models during the three periods. On account of the increase of costs during the high oil price period, it would be better for banks to use ARCHtype models to manage market risk in order not to worsen the profitability.
On the other hand, there is also an interesting phenomenon that the amounts of VaRs are independent of economic situations as illustrated in Figure 4. That is different from common sense that the worse the economic situation is, the larger VaRs are to prevent from bankruptcy.

Backtesting Method.
Given a VaR at the 1 percent level left-tail over daily horizon for a total of N days, one can count how many times the actual loss exceeds one day's VaR. Define n as the number of exceptions and n/N as the exception rate. The null hypothesis is that a given confidence level for losses is the true probability. Kupiec [24] approximates 95 percent confidence regions for such a test. The unconditional coverage is defined by the log likelihood ratio: which is distributed chi-square with one degree of freedom under the null hypothesis that d is the true probability (VaR confidence level). Also, we would reject the null hypothesis if LR uc > 3.84 at a 95% confidence level. As shown in Table 5, Panel A illustrates that the null hypothesis is rejected in AR(1)-ARCH(5), AR(1)-GARCH(1,1), AR(1)-GARCH (1,4), and AR(1)-EGARCH (1,1) models for RUD notes in the whole and the high oil price periods. It implies that these models are not suitable for a 1% significance level in the whole period and the high oil price period while 1% is the true probability for AR(1)-ARCH (5)   null hypothesis in AR(1)-GARCH(2,5) and AR(1)-ARCH(5) models for SPO notes in the whole period and in the low oil price period, respectively, whereas both AR(1)-GARCH (2,5) and AR(1)-ARCH(5) models are fit for a 1% significance level during the high oil price period. As for DJQ notes illustrated in Panel C, all of selected models are not fit for a 1% significance level in the low oil price period. However, one accepts the null hypothesis for AR(1)-GARCH(2,1) and EGARCH-type models in the whole period and the high oil price period.

Multivariate Extension of Diebold and Mariano Test.
Diebold and Mariano [19] consider two sequences of forecasts of a time series and propose some statistics to test equal predictive accuracy of two forecasting procedures for nested models. Alternatively, Mariano and Preve [23] provide a multivariate extension of the Diebold and Mariano test to evaluate a number of nonnested models that are to be compared in terms of a general loss. In the Mariano and Preve [23] model, the null hypothesis demonstrates equal predictive accuracy for the following predictive models: H 0 : E g e 1,t = E g e 2,t = · · · = E g e k+1,t , where {e i,t } are (k + 1) time series of forecast errors from (k + 1) alternative methods for i = 1, 2, . . . , (k + 1). That denotes the expected value of loss function of the ith forecast. The statistics is a chi-square distribution with k degrees of freedom. The equal accracy null hypothesis implies zero difference of mean squared error for competing forecasts. Let m = g(e j,t ) − g(e j+1,t ) for all j = 1,2,. . . , k. It is convenient to use the vector notation, M t = (m 1,t ,m 2,t ,. . .,m k,t ). Hence, the hypothesis of (10) can be written as E[M t ] = 0. As for the two competing models, the null hypothesis is E[g(e i,t )] = E [g(e j,t )] for i / = j and the statistics is χ 2 (1). Following the suggestions of past studies that GARCH model tends to have better performance in evaluating volatility, this paper uses the GARCH model to be a benchmark to compete with the other time series models selected. It is worthwhile noticing that the square errors of volatility are the gap between actual volatility (measured by a squared daily return) and the volatility generated by models.
As shown in Table 6, Panel A demonstrates that there is no significant difference in the accuracy of the couple competing models for RUD notes during the whole period when the benchmark is based on the GARCH-type models. The constant coefficient are not significantly positive in AR(1)-ARCH (5) and not significantly negative in AR(1)-GARCH(1,4) and AR(1)-EGARCH(1,1). It implies that the performances of AR(1)-GARCH(1,1) are not significantly worse than those of AR(1)-ARCH(5), while not significantly better than those of AR(1)-EGARCH(1,1) and AR (1) 4) is not significantly worse than the other selected models for RUD notes in the whole period. Therefore, as for RUD notes, there is no forecasting model that consistently has a superior performance in the whole period. Following the vein, the GARCH-type models do not significantly perform a better accuracy than AR(1)-ARCH(5) and AR(1)-EGARCH(1,1) in the low oil price period; their performance is not significantly better than that of AR(1)-ARCH(5) model but not significantly worse than that of AR(1)-EGARCH(1,1) in the high oil price period. As for SPO, the GARCH-type models are not significantly worse than the other models in the whole period; it is not significantly better than AR(1)-ARCH(5) but not significantly worse than AR(1)-EGARCH(1,1) in the low oil price period. GARCH-type models are not significantly better than both AR(1)-ARCH(5) and AR(1)-EGARCH(1,1) models during the high oil period. Hence the accuracy of AR(1)-ARCH(5) model is relatively better than the GARCH-type models during both low oil price and high oil price periods.
Return to Table 3. We apply RMSE and MAE to evaluate the accuracy of selected models. If the RMSE or MAE of a model is low, it stands for the accuracy of the model being better than the other.
As for RUD notes, one can find that the RMSEs (root mean squared errors) of AR(1)-ARCH(5), AR(1)-GARCH(1,1), and AR(1)-GARCH (1,4) and the MAE of AR(1)-EGARCH(1,1) are the smallest in the whole period; the RMSE of AR(1)-EGARCH(1,1) and the MAE of AR(1)-ARCH(5) are the lowest in the low oil price period; the RMSE and MAE of the GARCH-type models are the minimum in the high oil price period.
Also for SPO notes, the RMSE and MAE of the GARCHtype models are the smallest in the whole period, while the RMSE and MAE are the minimum in AR(1)-ARCH(5) model during the low oil price period. In the high oil price period, AR(1)-ARCH(5) and GARCH-type models have the lowest RMSE and MAE among the other selected models.
There is a common phenomenon for DJQ notes that the RMSE of EGARCH-type models is the lowest in any period. On the other hand, the accuracy of the GARCH-type models measured by MAE is better in both low oil price and high oil price periods.
Finally, we summarize the overall results in terms of backtesting method, the multivariate extension of Diebold and Mariano test, MAE, and MSE as illustrated in Table 7. There seems to be no forecasting model that consistently has superior performance based on these criteria. However, Table 7 reports that in spite of RUD, SPO, and DJQ notes, the frequency of the better performance for the GARCHtype models is higher than that of the ARCH model and the EGARCH-type models in the whole period, whereas it is not in the low price period and the high oil price periods. The accuracy of EGARCH models seems to be worse than that of the other models for the index-linked and principalprotected notes in the whole period, the low oil price period, and the high oil price period

Conclusion
Because VaR is a function of volatility, volatility forecast is the main issue of VaR. In this paper, using backtesting method, the multivariate extension of Diebold and Mariano test, RMSE, and MAE, we find that there seems to be no forecasting model that consistently has superior performance based on these criteria. But, the performance of the GARCHtype models is probably better than that of ARCH and EGARCH in the whole period, while it is not in the low price period and the high oil price periods. Despite in the low oil price and the high oil price period, ARCH model seems to have superior performance for stock index-linked and principal-protected notes.