A Note on the Performance of Biased Estimators with Autocorrelated Errors

It is a well-established fact in regression analysis that multicollinearity and autocorrelated errors have adverse effects on the properties of the least squares estimator. Huang and Yang (2015) and Chandra and Tyagi (2016) studied the PCTP estimator and the r − (k, d) class estimator, respectively, to deal with both problems simultaneously and compared their performances with the estimators obtained as their special cases. However, to the best of our knowledge, the performance of both estimators has not been compared so far. Hence, this paper is intended to compare the performance of these two estimators under mean squared error (MSE) matrix criterion. Further, a simulation study is conducted to evaluate superiority of the r − (k, d) class estimator over the PCTP estimator by means of percentage relative efficiency. Furthermore, two numerical examples have been given to illustrate the performance of the estimators.


Introduction
Let us consider a linear regression model as where  is an  × 1 vector of observations on dependent variable,  is an  ×  full column rank matrix of observations on  explanatory variables,  is a  × 1 vector of unknown regression coefficients, and  is an  × 1 vector of disturbance term with mean vector 0 and covariance matrix  2   .Ordinary least squares estimator (OLSE) is one of the most widely used estimator for , given as In the presence of multicollinearity among explanatory variables, OLSE becomes unstable and shows undesirable properties, such as inflated variance, wide confidence intervals which leads to wrong inferences and sometimes it even produces wrong signs of the estimates.
Numerous alternative methods of estimation have been designed to lower the effects of multicollinearity in literature.
For instance, Stein [1] proposed stein estimator; Hoerl and Kennard [2,3] introduced the technique of ordinary ridge regression estimator (ORRE); Massy [4] suggested principal component regression estimator (PCRE) to deal with the problem.Several authors combined two techniques of estimation in the hope that the combination will contain the advantages of the both.Baye and Parker [5] gave  −  class estimator by combining the PCRE and the ORRE, which includes the OLSE, ORRE, and PCRE as special cases.Nomura and Ohkubo [6] obtained conditions for dominance of the  −  class estimator over its special cases under mean squared error (MSE) criterion.Liu [7] gave an estimator by combining the advantages of the stein and ORRE, known as Liu estimator (LE).Kac ¸iranlar and Sakallıoglu [8] proposed  −  class estimator which is a combination of the LE and PCRE and showed the superiority of the  −  class estimator over the OLSE, LE, and PCRE.Özkale and Kac ¸ıranlar [9] proposed two-parameter estimator (TPE) by utilizing the advantages of the ORRE and LE and obtained necessary and sufficient condition for dominance of the TPE over the OLSE in MSE matrix sense.Further, Yang and Chang [10] also combined the ORR and Liu estimator in a different way and introduced an another two-parameter estimator (ATPE) and 2 International Journal of Mathematics and Mathematical Sciences derived necessary and sufficient conditions for superiority of the ATPE over OLSE, ORRE, LE, and TPE under MSE matrix criterion.Özkale [11] put forward a general class of estimators,  − (, ) class estimator which is a mingle of the TPE [9] and PCRE; they evaluated the performance of the  − (, ) class estimator under MSE criterion.Chang and Yang [12] suggested another general class of estimators by merging the PCRE and ATPE [10] named as principal component twoparameter estimator (PCTPE) and analyzed its performance under MSE matrix sense.
In applied work, it is quite common to have autocorrelation in error terms; that is, cov() =  2 Ω, where Ω is a known symmetric positive definite (p.d.)  ×  matrix and it is well known to statisticians that autocorrelated errors reduce the efficiency of the OLSE.Now, since Ω is a symmetrical positive definite matrix there exists an orthogonal matrix  such that   = Ω.On premultiplying model (1) by  −1 , we have (3) Note that ( * ) = 0 and cov( * ) =  2 .
Further, the rest of the paper is organized as follows: the necessary and sufficient condition for dominance of the PCTP estimator over the  − (, ) class estimator under the MSE matrix criterion has been derived in Section 2. Section 3 is devoted to simulation study to compare these estimators under MSE criterion.Some methods of selection of the unknown biasing parameters have been given in Section 4. Section 5 includes two numerical examples.Finally, the paper is summed up in Section 6 with some concluding remarks.

MSE Matrix Comparison of β𝑟 (𝑘,𝑑) and β𝑟 (𝑘,𝑑)
The MSE matrix criterion is a strong and one of the most widely used criteria for comparison of the estimators.Let β be an estimator of ; then the expression for the MSE matrix is given as where cov( β ) and Bias( β ) are the covariance matrix and bias vector of β .From ( 5) and ( 6), the covariance matrices and bias vectors of the −(, ) estimator and PCTP estimator can be obtained as where . Thus, the MSE matrices of the estimators can be given as To compare the performance of these estimators, the difference of the MSE matrices can be obtained as where On further simplification,  can be written as It is easy to note that  is positive definite.For the convenience of the derivation of the dominance conditions, we state the following Lemma.
From the expressions in ( 5) and ( 6), it is easy to verify that the  − (, ) class estimator and the PCTP estimator can be written as β (, ) =  1 βGLS and β (, ) =  2 βGLS , where Further, it is evident from ( 12) that  = cov( β (, )) − cov( β (, )) is a positive definite matrix.Thus from the above lemma, Δ  ≥ 0 if and only if Hence, the comparison under MSE matrix can be concluded in the following theorem.

Selection of 𝑘 and 𝑑
It is an important problem to find optimum value of the biasing parameters.A general approach to select an optimum value of the biasing parameters is to minimize the scalar MSE of the estimator.

For β𝑟 (𝑘, 𝑑).
The scalar MSE of the  − (, ) class estimator can be obtained by taking trace of the MSE matrix in (9), which is given as where   = ith component of  =   .The optimum value of () for a fixed () and  can be obtained by differentiating  1 (, , ) with respect to () and equating it to zero.Further, first derivative of  1 (, , ) with respect to  for fixed  and  is obtained as

International Journal of Mathematics and Mathematical Sciences
On equating to zero, we get the value of  for the  − (, ) class estimator as By taking harmonic mean as suggested by Hoerl et al. [27] and arithmetic mean and geometric mean [28] of the values in ( 16), we propose the following estimators: , Further, the positiveness of  1 can be ensured when we have  < min =1,2,..., { 2  /( 2 /  +  2  )} =  1 (say).It can be noted that when  = min =1,2,..., { 2  /( 2 /  +  2  )},  1 GM is not defined and  1 GM will give zero.This way we can choose a value of  satisfying  <  1 and further the value of  is obtained by replacing  in (17).
Alternatively, for fixed  and , the optimum value of  for the  − (, ) class estimator by minimizing  1 (, , ) with respect to  is obtained as Clearly,  1 opt is positive when  >  2 / 2 min .Hence, we can choose a value of  >  2 / 2 min and making use of this value we can find optimum value  1 opt .

For β𝑟 (𝑘, 𝑑).
The scalar MSE of the PCTP estimator obtained by taking trace of the MSE matrix in ( 10) is given as The first order derivative of  2 (, , ) is obtained as The optimum value of  for the PCTP estimator is obtained as Since  2 depends on , following Hoerl et al. [27] and Kibria [28], we propose the following estimators: Further, the positiveness of  2 can be ensured when we have This way we can chose a value of  satisfying  >  2 for the PCTP estimator and the value of  is then obtained by replacing  in (22).
Alternatively, for fixed  and , the optimum values of  for the PCTP estimator by minimizing  2 (, , ) with respect to  are obtained as When When  2 −  2  < 0 for all  = 1, 2, . . ., , 0 <  2 opt < 1 for 0 <  <  2 / 2 max .Further, the values of  and  can be easily obtained by replacing the unknown parameters  2 and  with their unbiased estimators.

Monte Carlo Study
In this section, we will evaluate the performance of the estimators through Monte Carlo simulation.Following McDonald and Galarneau [29] and Gibbons [30],  matrix has been generated as follows: where   are generated from standard normal pseudorandom numbers and   's are generated such that the correlation between any pair of -variables is  2 .In this study, we consider the values of  to be 0.90, 0.95, and 0.99.Following McDonald and Galarneau [29], Gibbons [30], Kibria [28], and others,  has been chosen as the normalized eigenvector corresponding to the largest eigenvalue of the    matrix.
The dependent variable  is obtained by Following Firinguetti [15], Judge et al. [31], and Chandra and Sarkar [24],  are generated from AR(1) process as where   are independent normal pseudorandom numbers with mean 0 and variance  2  and  is autoregressive coefficient such that || < 1.The covariance matrix Ω for AR (1) errors is given by The value of  is decided by a scree plot which is drawn between eigenvalues and components (see Johnson and Wichern [32]).In this simulation we chose  = 20, 50, 100,  = 5, 10,  2  = 0.1, 1, 10,  = 0, 0.3, 0.9,  = 0.1, 0.5, 0.9, 2, and  = 0.1, 0.5, 0.9.Then the experiment is repeated 2000 times by generating errors in every repetition and estimated MSE (EMSE) is calculated by the following formula: where β() is the estimated value of  in ith iteration.To compare the performances of the estimators, percentage relative efficiency of the  − (, ) class estimator over the PCTP estimator has been calculated as follows: For brevity, we have reported some selected results in Tables 1-3, where  1 ,  2 , and  3 give percent relative efficiency of the −(, ) class estimator over the PCTP estimator when  takes values 0.90, 0.95, and 0.99, respectively.
Since all the values of percent relative efficiency shown in Tables 1-3 are positive, it implies that the  − (, ) class estimator is more efficient than the PCTP estimator in all the cases considered in this study.However, when we examine the behavior of the percent relative efficiency for different parameters considered here, it is observed that the parameters , , , and  affect the percent relative efficiency negatively.That is, when these parameters take larger values, the percent relative efficiency decreases and approaches zero.Alternatively, we can say that there are some values of the parameters for which the value of percent relative efficiency is so low that it can be considered that both the estimators perform equally well, for example, when  2  = 0.1,  = 20, 50, 100, and  = 10 and  2  = 0.1,  = 20, 50, 100,  = 5, and  = 0.9 for most of the values of  and .

Numerical Example
In this section we examine the performance of the two estimators, namely, the −(, ) class estimator and the PCTP estimator in MSE sense using two numerical examples; one is for US GDP data and the other is the famous Hald data [33].
Example 1 (US GDP Data).The quarterly US data on GDP growth (), personal disposable income ( 1 ), personal consumption expenditure ( 2 ), and corporate tax after profits ( 3 ) for the years 1970-1991 have been taken from Gujarati [34].The data has also been used by Chandra and Sarkar [24].The variables are standardized and the eigenvalues of    matrix are obtained as 324.6527396, 21.8742084, 1.2365875, and 0.2364645, which shows high multicollinearity in the data.Further, the value of the DW statistic for the data is found to be 0.4784, which indicates the presence of positive autocorrelation at the significance level of 0.05 with the two limits of the critical value being   = 1.429 and   = 1.611 for  = 88.The error structure follows AR(1) process with estimated  to be 0.7530 and  2  = 0.001118.Thus the Ω matrix can be constructed using (28).The condition number of   Ω −1  is obtained to be 246.3951which indicates high multicollinearity in the data.Although the GLSE is unstable in the presence of multicollinearity, several studies have suggested that the estimate of variance based on least squares estimator is superior to the estimates based on other shrinkage estimators, for instance, see Ohtani [35], Dube and Chandra [36], and Ünal [37].Hence, we have estimated the value of  2 by using the GLSE of , which comes out to be 0.00292.The eigenvalues of   Ω −1  are  1 = 23.78081, 2 = 3.0276,  3 = 0.2420, and  4 = 0.0965.We chose  = 2 which accounts for 99.57% of variation in the data.Now, we chose a value of  to be 0.2505 such that  >  2 / 2 min = 0.1505 for the  − (, ) class estimator and thus the optimum value  1 opt is obtained to be 0.4286.Further, since estimated  2 is smaller than  2  = (0.258642223, 0.0194366),  is selected as 0.0056 which belongs to the interval 0 <  <  2 / 2 max = 0.01131 and hence  2 opt is found to be 0.8501.The MSEs of the  − (, ) class estimator and the PCTP estimator for the obtained optimum values are 0.1396499 and 0.1396482, respectively.Clearly, the values are almost the same which suggests that both the estimators perform equally well for the corresponding optimum values.However, the difference can       Further, the estimated MSEs of the  − (, ) class and the PCTP estimators with respect to  for fixed  = 0.25, 0.5 and with respect to  for  = 0.7, 0.95 are represented in Figure 1.The values of  and  are selected so that we can observe the behavior of estimators for  and  near and far from their respective optimum values.The figures depict superiority of the  − (, ) estimator over the PCTP estimator for larger range of  and .Further, it can be seen from Figure 1(a) that the PCTP estimator starts dominating the  − (, ) class estimator after a point, whereas for  = 0.5 in Figure 1(b) we do not see a superiority of the PCTP estimator in the whole range of 0 <  < 1.Similarly, we get a range of  in which the PCTP estimator dominates the  − (, ) class estimator for  = 0.95 and not for  = 0.7; see Figures 1(c) and 1(d).
Example 2 (Hald Data).Now, let us consider the data set on Portland cement originally due to Woods et al. [38] and then analyzed by Hald [33] and known as Hald data.The data is an outcome of an experiment conducted to investigate the heat evolved during setting and hardening of Portland cements of varied composition and the dependence of this heat on the percentages of four compounds in the clinkers from which the cement was produced.The data includes the heat evolved in calories per gram of cement () as dependent variable and four ingredients: tricalcium aluminate ( 1 ), tetracalcium silicate ( 2 ), tetracalcium aluminoferrite ( 3 ), and dicalcium silicate ( 4 ) as explanatory variables.
Following Özkale [11], the variables are standardized so that the    matrix forms a correlation matrix and the eigenvalues obtained are  1 = 2.235704,  2 = 1.576066,  3 = 0.186606, and  4 = 0.001623.The value of Durbin-Watson test comes out to be 2.052597 resulting in the conclusion of no autocorrelation in error term at 5% level of significance with   = 0.574 and   = 2.094 for  = 13; hence we can consider Ω =   .The estimated value of  2 is 0.00196 and the value of  has been chosen to be 2, which accounts for 95.29% of variation in data.The optimum value of  for a selected value of  is chosen for the  − (, ) class and PCTP estimators.For the  − (, ) class estimator, we chose value of  as 1.322 such that  >  2 /   hence we chose a value of  in the range max =1,2,..., {  ( 2 −  2  )/(  + 1) 2  } <  <  2 / 2 max .The value of lower bound is obtained as 0.000000574, which is approximately 0 and the upper bound is 0.004516; hence we chose a value  as 0.0022 and for this value  2 opt for the PCTP estimator is obtained as 0.9854.The MSEs of the  − (, ) class and PCTP estimators are obtained to be the same up to 7 decimal places, that is, 0.3704072, indicating the same performance at the optimum values.
Moreover, the performance of both estimators for various  and  is represented in Figures 2(a)-2(d).Clearly, the  − (, ) class estimator is exhibiting better performance for larger range of  and .However, a careful examination of Figures 2(a) and 2(d) suggests that there may be some points of  and  where the PCTP may perform better.Figures 2(b) and 2(c) show that there is no value of  and , respectively, for  = 0.5 and  = 0.7.
Looking at the results of both examples we observe that the  − (, ) class estimator performs better than the PCTP estimator in scalar MSE sense for most of the values of  and  under study.However, Figures 1(a

Conclusion
In this paper we have examined the performance of two biased estimators in the presence of multicollinearity with autocorrelated errors which include the same number of unknown parameters with same range.Further, a method of selection of  and  in both estimators has been suggested in terms of minimizing scalar MSE.The conditions of dominance of the PCTP estimator over the  − (, ) class estimator have been derived using MSE matrix as comparison criterion.Further, we have performed a simulation study and the percentage relative efficiency of the  − (, ) class estimator over the PCTP estimator has been evaluated.Moreover, two numerical examples are considered to compare the two estimators.The simulation study suggests that for all the parametric conditions considered here the  − (, ) class estimator performs better than the PCTP estimator in scalar MSE sense.The numerical examples give the results in favor of the simulation results; that is, the  − (, ) class estimator performed better than the PCTP estimator under scalar MSE criterion for most of the values of  and .However, for optimum values of  and  the performance of both estimators is similar and the superiority of the PCTP estimator may be seen after the fourth or fifth decimal places.

Table 1 : 20 and
Percent relative efficiency of the  − (, ) class estimator over the PCTP estimator for  =

Figure 1 :
Figure 1: Estimated mean squared error of  − (, ) class and the PCTP estimators for US GDP data.

Figure 2 :
Figure 2: Estimated mean squared error of  − (, ) class and the PCTP estimators for Hald data.

Table 2 :
Percent relative efficiency of the  − (, ) class estimator over the PCTP estimator for  = 50 2 min = 1.222, and for this value of  the optimum value  1 opt is obtained as 0.9899.Further, we obtain  2  = (0.4348143, 0.0016061) which indicates that  2 is larger than the second value of  2  , International Journal of Mathematics and Mathematical Sciences